Jul 07 15:41:51.673360 ip-10-0-131-221 systemd[1]: Starting Kubernetes Kubelet... Jul 07 15:41:52.095484 ip-10-0-131-221 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 15:41:52.095484 ip-10-0-131-221 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 15:41:52.095484 ip-10-0-131-221 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 15:41:52.095484 ip-10-0-131-221 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 15:41:52.095484 ip-10-0-131-221 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 15:41:52.096368 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.096229 2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 15:41:52.101977 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.101951 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:41:52.101977 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.101971 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:41:52.101977 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.101977 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:41:52.101977 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.101981 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:41:52.101977 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.101985 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:41:52.102303 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.101989 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:41:52.102303 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.101993 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:41:52.102303 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.101997 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:41:52.102303 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102000 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:41:52.102303 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102004 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:41:52.102303 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102009 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:41:52.102303 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102012 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:41:52.102303 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102016 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:41:52.102303 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102020 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:41:52.102303 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102023 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:41:52.102303 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102027 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:41:52.102303 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102044 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:41:52.102303 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102049 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:41:52.102303 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102054 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:41:52.102303 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102057 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:41:52.102303 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102061 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:41:52.102303 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102067 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:41:52.102303 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102073 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:41:52.102303 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102077 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:41:52.102303 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102082 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:41:52.103145 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102086 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:41:52.103145 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102091 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:41:52.103145 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102095 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:41:52.103145 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102105 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:41:52.103145 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102109 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:41:52.103145 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102113 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:41:52.103145 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102117 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:41:52.103145 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102120 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:41:52.103145 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102124 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:41:52.103145 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102128 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:41:52.103145 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102132 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:41:52.103145 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102136 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:41:52.103145 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102140 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:41:52.103145 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102146 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:41:52.103145 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102155 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:41:52.103145 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102162 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:41:52.103145 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102167 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:41:52.103145 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102171 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:41:52.103145 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102175 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:41:52.103915 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102179 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:41:52.103915 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102183 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:41:52.103915 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102188 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:41:52.103915 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102192 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:41:52.103915 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102197 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:41:52.103915 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102201 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:41:52.103915 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102206 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:41:52.103915 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102211 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:41:52.103915 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102215 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:41:52.103915 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102219 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:41:52.103915 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102224 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:41:52.103915 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102228 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:41:52.103915 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102232 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:41:52.103915 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102238 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:41:52.103915 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102243 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:41:52.103915 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102247 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:41:52.103915 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102251 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:41:52.103915 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102255 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:41:52.103915 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102259 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:41:52.103915 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102264 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:41:52.104775 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102269 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:41:52.104775 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102273 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:41:52.104775 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102277 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:41:52.104775 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102281 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:41:52.104775 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102286 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:41:52.104775 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102290 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:41:52.104775 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102295 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:41:52.104775 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102299 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:41:52.104775 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102303 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:41:52.104775 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102308 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:41:52.104775 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102312 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:41:52.104775 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102317 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:41:52.104775 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102321 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:41:52.104775 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102326 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:41:52.104775 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102331 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:41:52.104775 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102335 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:41:52.104775 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102339 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:41:52.104775 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102344 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:41:52.104775 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102348 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:41:52.104775 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102353 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:41:52.105641 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102357 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:41:52.105641 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102361 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:41:52.105641 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102366 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:41:52.105641 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102370 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:41:52.105641 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102374 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:41:52.105641 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102379 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:41:52.105641 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102383 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:41:52.105641 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102387 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:41:52.105641 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102394 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:41:52.105641 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102399 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:41:52.105641 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102404 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:41:52.105641 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102408 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:41:52.105641 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102413 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:41:52.105641 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102417 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:41:52.105641 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.102421 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:41:52.105641 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103723 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:41:52.105641 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103735 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:41:52.105641 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103740 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:41:52.105641 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103746 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:41:52.106531 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103750 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:41:52.106531 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103755 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:41:52.106531 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103759 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:41:52.106531 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103763 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:41:52.106531 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103767 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:41:52.106531 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103772 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:41:52.106531 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103776 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:41:52.106531 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103780 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:41:52.106531 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103784 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:41:52.106531 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103789 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:41:52.106531 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103793 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:41:52.106531 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103797 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:41:52.106531 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103801 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:41:52.106531 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103807 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:41:52.106531 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103812 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:41:52.106531 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103816 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:41:52.106531 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103820 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:41:52.106531 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103824 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:41:52.106531 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103828 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:41:52.107211 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103833 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:41:52.107211 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103837 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:41:52.107211 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103844 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:41:52.107211 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103849 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:41:52.107211 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103855 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:41:52.107211 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103860 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:41:52.107211 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103865 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:41:52.107211 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103869 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:41:52.107211 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103873 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:41:52.107211 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103877 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:41:52.107211 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103882 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:41:52.107211 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103887 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:41:52.107211 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103891 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:41:52.107211 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103896 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:41:52.107211 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103900 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:41:52.107211 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103904 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:41:52.107211 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103909 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:41:52.107211 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103913 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:41:52.107211 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103917 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:41:52.107211 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103920 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:41:52.107818 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103924 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:41:52.107818 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103928 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:41:52.107818 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103932 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:41:52.107818 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103937 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:41:52.107818 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103941 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:41:52.107818 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103945 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:41:52.107818 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103950 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:41:52.107818 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103955 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:41:52.107818 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103959 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:41:52.107818 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103963 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:41:52.107818 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103967 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:41:52.107818 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103972 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:41:52.107818 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103976 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:41:52.107818 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103980 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:41:52.107818 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103984 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:41:52.107818 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103989 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:41:52.107818 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.103997 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:41:52.107818 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104002 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:41:52.107818 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104006 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:41:52.108307 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104010 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:41:52.108307 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104014 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:41:52.108307 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104019 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:41:52.108307 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104023 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:41:52.108307 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104044 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:41:52.108307 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104050 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:41:52.108307 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104053 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:41:52.108307 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104058 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:41:52.108307 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104062 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:41:52.108307 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104066 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:41:52.108307 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104070 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:41:52.108307 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104075 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:41:52.108307 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104079 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:41:52.108307 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104083 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:41:52.108307 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104087 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:41:52.108307 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104091 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:41:52.108307 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104095 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:41:52.108307 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104099 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:41:52.108307 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104104 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:41:52.108307 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104108 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:41:52.108876 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104113 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:41:52.108876 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104118 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:41:52.108876 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104123 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:41:52.108876 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104128 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:41:52.108876 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104135 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:41:52.108876 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104142 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:41:52.108876 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104146 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:41:52.108876 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104150 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:41:52.108876 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104154 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:41:52.108876 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104170 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:41:52.108876 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104177 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:41:52.108876 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104182 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:41:52.108876 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104186 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:41:52.108876 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104191 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:41:52.108876 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104195 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:41:52.108876 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104200 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:41:52.108876 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104205 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:41:52.108876 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104855 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:41:52.108876 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104865 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:41:52.109495 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104870 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:41:52.109495 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104874 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:41:52.109495 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104879 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:41:52.109495 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104883 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:41:52.109495 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104889 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:41:52.109495 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104894 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:41:52.109495 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104899 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:41:52.109495 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104903 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:41:52.109495 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104908 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:41:52.109495 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104912 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:41:52.109495 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104916 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:41:52.109495 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104921 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:41:52.109495 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104924 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:41:52.109495 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104929 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:41:52.109495 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104934 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:41:52.109495 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104938 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:41:52.109495 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104943 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:41:52.109495 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104947 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:41:52.109495 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104951 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:41:52.109495 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104955 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:41:52.109983 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104961 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:41:52.109983 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104968 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:41:52.109983 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104978 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:41:52.109983 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104983 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:41:52.109983 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104988 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:41:52.109983 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104993 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:41:52.109983 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.104997 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:41:52.109983 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105002 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:41:52.109983 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105007 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:41:52.109983 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105011 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:41:52.109983 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105015 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:41:52.109983 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105020 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:41:52.109983 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105024 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:41:52.109983 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105049 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:41:52.109983 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105056 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:41:52.109983 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105061 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:41:52.109983 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105065 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:41:52.109983 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105069 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:41:52.109983 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105073 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:41:52.110560 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105077 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:41:52.110560 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105081 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:41:52.110560 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105084 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:41:52.110560 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105089 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:41:52.110560 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105093 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:41:52.110560 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105097 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:41:52.110560 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105101 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:41:52.110560 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105105 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:41:52.110560 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105110 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:41:52.110560 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105114 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:41:52.110560 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105118 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:41:52.110560 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105122 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:41:52.110560 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105126 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:41:52.110560 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105130 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:41:52.110560 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105134 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:41:52.110560 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105141 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:41:52.110560 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105145 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:41:52.110560 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105150 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:41:52.110560 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105154 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:41:52.110560 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105158 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:41:52.111055 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105162 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:41:52.111055 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105166 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:41:52.111055 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105170 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:41:52.111055 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105175 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:41:52.111055 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105181 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:41:52.111055 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105187 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:41:52.111055 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105191 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:41:52.111055 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105196 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:41:52.111055 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105200 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:41:52.111055 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105204 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:41:52.111055 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105208 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:41:52.111055 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105212 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:41:52.111055 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105217 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:41:52.111055 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105221 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:41:52.111055 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105225 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:41:52.111055 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105229 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:41:52.111055 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105233 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:41:52.111055 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105238 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:41:52.111055 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105241 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:41:52.111055 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105245 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:41:52.111538 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105250 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:41:52.111538 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105255 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:41:52.111538 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105259 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:41:52.111538 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105263 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:41:52.111538 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105268 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:41:52.111538 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105272 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:41:52.111538 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105276 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:41:52.111538 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105282 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:41:52.111538 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105286 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:41:52.111538 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105293 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:41:52.111538 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105297 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:41:52.111538 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105301 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:41:52.111538 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105306 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:41:52.111538 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105310 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:41:52.111538 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105314 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:41:52.111538 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105318 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:41:52.111538 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105322 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:41:52.111538 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.105326 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:41:52.111538 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105434 2575 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 15:41:52.111538 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105445 2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 15:41:52.112013 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105455 2575 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 15:41:52.112013 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105462 2575 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 15:41:52.112013 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105469 2575 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 15:41:52.112013 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105474 2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 15:41:52.112013 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105480 2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 15:41:52.112013 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105488 2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 15:41:52.112013 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105492 2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 15:41:52.112013 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105497 2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 15:41:52.112013 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105503 2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 15:41:52.112013 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105508 2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 15:41:52.112013 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105513 2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 15:41:52.112013 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105518 2575 flags.go:64] FLAG: --cgroup-root="" Jul 07 15:41:52.112013 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105522 2575 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 15:41:52.112013 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105527 2575 flags.go:64] FLAG: --client-ca-file="" Jul 07 15:41:52.112013 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105534 2575 flags.go:64] FLAG: --cloud-provider="external" Jul 07 15:41:52.112013 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105539 2575 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 15:41:52.112013 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105546 2575 flags.go:64] FLAG: --cluster-domain="" Jul 07 15:41:52.112013 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105550 2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 15:41:52.112013 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105556 2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 15:41:52.112013 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105563 2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 15:41:52.112013 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105568 2575 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 15:41:52.112013 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105575 2575 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 15:41:52.112013 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105579 2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 15:41:52.112579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105584 2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 15:41:52.112579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105590 2575 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 15:41:52.112579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105595 2575 flags.go:64] FLAG: --contention-profiling="false" Jul 07 15:41:52.112579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105599 2575 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 15:41:52.112579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105604 2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 15:41:52.112579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105609 2575 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 15:41:52.112579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105614 2575 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 15:41:52.112579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105620 2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 15:41:52.112579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105626 2575 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 15:41:52.112579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105630 2575 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 15:41:52.112579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105635 2575 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 15:41:52.112579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105639 2575 flags.go:64] FLAG: --enable-server="true" Jul 07 15:41:52.112579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105644 2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 15:41:52.112579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105651 2575 flags.go:64] FLAG: --event-burst="100" Jul 07 15:41:52.112579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105655 2575 flags.go:64] FLAG: --event-qps="50" Jul 07 15:41:52.112579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105659 2575 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 15:41:52.112579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105664 2575 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 15:41:52.112579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105669 2575 flags.go:64] FLAG: --eviction-hard="" Jul 07 15:41:52.112579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105675 2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 15:41:52.112579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105680 2575 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 15:41:52.112579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105685 2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 15:41:52.112579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105689 2575 flags.go:64] FLAG: --eviction-soft="" Jul 07 15:41:52.112579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105694 2575 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 15:41:52.112579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105699 2575 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 15:41:52.112579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105705 2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 15:41:52.113282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105709 2575 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 15:41:52.113282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105714 2575 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 15:41:52.113282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105718 2575 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 15:41:52.113282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105726 2575 flags.go:64] FLAG: --feature-gates="" Jul 07 15:41:52.113282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105740 2575 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 15:41:52.113282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105745 2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 15:41:52.113282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105750 2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 15:41:52.113282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105755 2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 15:41:52.113282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105760 2575 flags.go:64] FLAG: --healthz-port="10248" Jul 07 15:41:52.113282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105764 2575 flags.go:64] FLAG: --help="false" Jul 07 15:41:52.113282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105769 2575 flags.go:64] FLAG: --hostname-override="ip-10-0-131-221.ec2.internal" Jul 07 15:41:52.113282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105774 2575 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 15:41:52.113282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105779 2575 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 15:41:52.113282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105783 2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 15:41:52.113282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105789 2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 15:41:52.113282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105795 2575 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 15:41:52.113282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105799 2575 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 15:41:52.113282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105803 2575 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 15:41:52.113282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105808 2575 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 15:41:52.113282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105812 2575 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 15:41:52.113282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105817 2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 15:41:52.113282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105822 2575 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 15:41:52.113282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105827 2575 flags.go:64] FLAG: --kube-reserved="" Jul 07 15:41:52.113282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105831 2575 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 15:41:52.113893 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105836 2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 15:41:52.113893 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105841 2575 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 15:41:52.113893 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105845 2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 15:41:52.113893 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105850 2575 flags.go:64] FLAG: --lock-file="" Jul 07 15:41:52.113893 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105855 2575 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 15:41:52.113893 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105860 2575 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 15:41:52.113893 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105865 2575 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 15:41:52.113893 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105874 2575 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 15:41:52.113893 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105880 2575 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 15:41:52.113893 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105885 2575 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 15:41:52.113893 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105890 2575 flags.go:64] FLAG: --logging-format="text" Jul 07 15:41:52.113893 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105896 2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 15:41:52.113893 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105902 2575 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 15:41:52.113893 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105906 2575 flags.go:64] FLAG: --manifest-url="" Jul 07 15:41:52.113893 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105911 2575 flags.go:64] FLAG: --manifest-url-header="" Jul 07 15:41:52.113893 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105918 2575 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 15:41:52.113893 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105923 2575 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 15:41:52.113893 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105929 2575 flags.go:64] FLAG: --max-pods="110" Jul 07 15:41:52.113893 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105934 2575 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 15:41:52.113893 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105939 2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 15:41:52.113893 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105943 2575 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 15:41:52.113893 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105948 2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 15:41:52.113893 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105953 2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 15:41:52.113893 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105957 2575 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 15:41:52.113893 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105962 2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 15:41:52.114508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105975 2575 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 15:41:52.114508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105979 2575 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 15:41:52.114508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105984 2575 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 15:41:52.114508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105989 2575 flags.go:64] FLAG: --pod-cidr="" Jul 07 15:41:52.114508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.105994 2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 15:41:52.114508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106002 2575 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 15:41:52.114508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106007 2575 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 15:41:52.114508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106012 2575 flags.go:64] FLAG: --pods-per-core="0" Jul 07 15:41:52.114508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106016 2575 flags.go:64] FLAG: --port="10250" Jul 07 15:41:52.114508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106021 2575 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 15:41:52.114508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106044 2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0cf975ad6df992a8d" Jul 07 15:41:52.114508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106050 2575 flags.go:64] FLAG: --qos-reserved="" Jul 07 15:41:52.114508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106055 2575 flags.go:64] FLAG: --read-only-port="10255" Jul 07 15:41:52.114508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106059 2575 flags.go:64] FLAG: --register-node="true" Jul 07 15:41:52.114508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106064 2575 flags.go:64] FLAG: --register-with-taints="" Jul 07 15:41:52.114508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106069 2575 flags.go:64] FLAG: --registry-burst="10" Jul 07 15:41:52.114508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106075 2575 flags.go:64] FLAG: --registry-qps="5" Jul 07 15:41:52.114508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106079 2575 flags.go:64] FLAG: --reserved-cpus="" Jul 07 15:41:52.114508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106086 2575 flags.go:64] FLAG: --reserved-memory="" Jul 07 15:41:52.114508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106092 2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 15:41:52.114508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106097 2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 15:41:52.114508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106102 2575 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 15:41:52.114508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106107 2575 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 15:41:52.114508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106111 2575 flags.go:64] FLAG: --runonce="false" Jul 07 15:41:52.114508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106116 2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 15:41:52.115133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106121 2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 15:41:52.115133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106126 2575 flags.go:64] FLAG: --seccomp-default="false" Jul 07 15:41:52.115133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106130 2575 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 15:41:52.115133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106135 2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 15:41:52.115133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106139 2575 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 15:41:52.115133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106144 2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 15:41:52.115133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106149 2575 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 15:41:52.115133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106153 2575 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 15:41:52.115133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106158 2575 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 15:41:52.115133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106162 2575 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 15:41:52.115133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106167 2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 15:41:52.115133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106172 2575 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 15:41:52.115133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106177 2575 flags.go:64] FLAG: --system-cgroups="" Jul 07 15:41:52.115133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106181 2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 15:41:52.115133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106190 2575 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 15:41:52.115133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106195 2575 flags.go:64] FLAG: --tls-cert-file="" Jul 07 15:41:52.115133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106199 2575 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 15:41:52.115133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106208 2575 flags.go:64] FLAG: --tls-min-version="" Jul 07 15:41:52.115133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106213 2575 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 15:41:52.115133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106218 2575 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 15:41:52.115133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106222 2575 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 15:41:52.115133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106227 2575 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 15:41:52.115133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106232 2575 flags.go:64] FLAG: --v="2" Jul 07 15:41:52.115133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106238 2575 flags.go:64] FLAG: --version="false" Jul 07 15:41:52.115133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106245 2575 flags.go:64] FLAG: --vmodule="" Jul 07 15:41:52.115734 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106253 2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 15:41:52.115734 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.106259 2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 15:41:52.115734 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106466 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:41:52.115734 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106473 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:41:52.115734 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106479 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:41:52.115734 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106485 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:41:52.115734 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106491 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:41:52.115734 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106495 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:41:52.115734 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106500 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:41:52.115734 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106505 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:41:52.115734 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106509 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:41:52.115734 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106513 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:41:52.115734 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106518 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:41:52.115734 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106522 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:41:52.115734 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106526 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:41:52.115734 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106530 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:41:52.115734 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106534 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:41:52.115734 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106538 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:41:52.115734 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106542 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:41:52.115734 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106546 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:41:52.116236 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106550 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:41:52.116236 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106554 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:41:52.116236 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106558 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:41:52.116236 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106562 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:41:52.116236 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106566 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:41:52.116236 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106573 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:41:52.116236 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106579 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:41:52.116236 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106585 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:41:52.116236 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106590 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:41:52.116236 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106594 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:41:52.116236 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106599 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:41:52.116236 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106604 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:41:52.116236 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106611 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:41:52.116236 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106616 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:41:52.116236 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106620 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:41:52.116236 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106625 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:41:52.116236 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106629 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:41:52.116236 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106633 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:41:52.116236 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106638 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:41:52.116715 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106642 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:41:52.116715 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106646 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:41:52.116715 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106651 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:41:52.116715 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106655 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:41:52.116715 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106659 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:41:52.116715 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106663 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:41:52.116715 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106668 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:41:52.116715 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106672 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:41:52.116715 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106676 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:41:52.116715 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106680 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:41:52.116715 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106686 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:41:52.116715 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106690 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:41:52.116715 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106695 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:41:52.116715 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106699 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:41:52.116715 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106703 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:41:52.116715 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106707 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:41:52.116715 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106711 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:41:52.116715 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106716 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:41:52.116715 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106721 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:41:52.116715 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106725 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:41:52.117240 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106730 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:41:52.117240 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106735 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:41:52.117240 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106739 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:41:52.117240 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106743 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:41:52.117240 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106748 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:41:52.117240 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106753 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:41:52.117240 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106759 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:41:52.117240 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106764 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:41:52.117240 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106768 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:41:52.117240 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106773 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:41:52.117240 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106777 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:41:52.117240 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106781 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:41:52.117240 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106785 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:41:52.117240 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106789 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:41:52.117240 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106794 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:41:52.117240 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106797 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:41:52.117240 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106801 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:41:52.117240 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106806 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:41:52.117240 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106812 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:41:52.117240 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106816 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:41:52.117743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106821 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:41:52.117743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106825 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:41:52.117743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106829 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:41:52.117743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106834 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:41:52.117743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106840 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:41:52.117743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106846 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:41:52.117743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106851 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:41:52.117743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106856 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:41:52.117743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106861 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:41:52.117743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106865 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:41:52.117743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106871 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:41:52.117743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106875 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:41:52.117743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106879 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:41:52.117743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106883 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:41:52.117743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106887 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:41:52.117743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106892 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:41:52.117743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106896 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:41:52.117743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106902 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:41:52.117743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106906 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:41:52.117743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106911 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:41:52.118256 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106915 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:41:52.118256 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.106920 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:41:52.118256 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.107561 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 15:41:52.118256 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.115611 2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 15:41:52.118256 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.115627 2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 15:41:52.118256 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.115640 2575 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 15:41:52.118256 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.115646 2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 15:41:52.118256 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115716 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:41:52.118256 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115722 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:41:52.118256 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115725 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:41:52.118256 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115728 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:41:52.118256 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115730 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:41:52.118256 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115733 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:41:52.118256 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115736 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:41:52.118256 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115739 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:41:52.118668 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115742 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:41:52.118668 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115745 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:41:52.118668 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115748 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:41:52.118668 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115751 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:41:52.118668 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115753 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:41:52.118668 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115756 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:41:52.118668 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115759 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:41:52.118668 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115762 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:41:52.118668 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115764 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:41:52.118668 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115767 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:41:52.118668 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115770 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:41:52.118668 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115772 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:41:52.118668 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115776 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:41:52.118668 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115778 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:41:52.118668 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115781 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:41:52.118668 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115783 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:41:52.118668 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115786 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:41:52.118668 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115788 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:41:52.118668 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115791 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:41:52.118668 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115794 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:41:52.119169 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115796 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:41:52.119169 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115800 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:41:52.119169 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115804 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:41:52.119169 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115806 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:41:52.119169 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115809 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:41:52.119169 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115813 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:41:52.119169 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115816 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:41:52.119169 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115820 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:41:52.119169 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115822 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:41:52.119169 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115825 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:41:52.119169 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115828 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:41:52.119169 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115830 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:41:52.119169 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115833 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:41:52.119169 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115835 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:41:52.119169 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115838 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:41:52.119169 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115840 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:41:52.119169 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115843 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:41:52.119169 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115845 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:41:52.119169 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115848 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:41:52.119635 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115850 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:41:52.119635 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115854 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:41:52.119635 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115858 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:41:52.119635 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115861 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:41:52.119635 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115863 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:41:52.119635 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115866 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:41:52.119635 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115868 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:41:52.119635 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115871 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:41:52.119635 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115873 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:41:52.119635 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115876 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:41:52.119635 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115878 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:41:52.119635 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115881 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:41:52.119635 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115883 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:41:52.119635 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115886 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:41:52.119635 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115891 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:41:52.119635 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115895 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:41:52.119635 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115898 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:41:52.119635 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115902 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:41:52.119635 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115904 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:41:52.120252 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115907 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:41:52.120252 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115910 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:41:52.120252 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115912 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:41:52.120252 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115915 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:41:52.120252 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115918 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:41:52.120252 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115920 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:41:52.120252 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115923 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:41:52.120252 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115925 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:41:52.120252 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115928 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:41:52.120252 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115930 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:41:52.120252 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115933 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:41:52.120252 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115936 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:41:52.120252 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115938 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:41:52.120252 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115941 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:41:52.120252 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115943 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:41:52.120252 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115946 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:41:52.120252 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115948 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:41:52.120252 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115951 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:41:52.120252 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115953 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:41:52.120252 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115956 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:41:52.120743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115958 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:41:52.120743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115961 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:41:52.120743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115963 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:41:52.120743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115966 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:41:52.120743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115968 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:41:52.120743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115971 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:41:52.120743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115973 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:41:52.120743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115977 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:41:52.120743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115980 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:41:52.120743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115983 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:41:52.120743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115985 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:41:52.120743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115988 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:41:52.120743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.115990 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:41:52.120743 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.115995 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 15:41:52.120743 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116138 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:41:52.121153 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116142 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:41:52.121153 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116145 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:41:52.121153 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116149 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:41:52.121153 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116152 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:41:52.121153 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116154 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:41:52.121153 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116157 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:41:52.121153 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116160 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:41:52.121153 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116162 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:41:52.121153 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116165 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:41:52.121153 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116167 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:41:52.121153 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116170 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:41:52.121153 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116172 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:41:52.121153 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116175 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:41:52.121153 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116177 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:41:52.121153 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116180 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:41:52.121153 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116182 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:41:52.121153 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116185 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:41:52.121153 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116187 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:41:52.121153 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116190 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:41:52.121153 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116192 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:41:52.121640 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116194 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:41:52.121640 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116197 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:41:52.121640 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116200 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:41:52.121640 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116202 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:41:52.121640 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116221 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:41:52.121640 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116226 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:41:52.121640 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116229 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:41:52.121640 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116232 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:41:52.121640 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116236 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:41:52.121640 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116240 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:41:52.121640 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116243 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:41:52.121640 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116247 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:41:52.121640 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116251 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:41:52.121640 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116254 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:41:52.121640 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116257 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:41:52.121640 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116259 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:41:52.121640 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116262 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:41:52.121640 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116264 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:41:52.121640 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116267 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:41:52.122122 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116269 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:41:52.122122 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116271 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:41:52.122122 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116274 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:41:52.122122 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116276 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:41:52.122122 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116279 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:41:52.122122 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116281 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:41:52.122122 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116284 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:41:52.122122 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116286 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:41:52.122122 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116289 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:41:52.122122 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116292 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:41:52.122122 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116294 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:41:52.122122 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116297 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:41:52.122122 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116299 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:41:52.122122 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116302 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:41:52.122122 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116304 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:41:52.122122 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116307 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:41:52.122122 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116309 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:41:52.122122 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116312 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:41:52.122122 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116315 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:41:52.122122 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116318 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:41:52.122594 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116320 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:41:52.122594 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116323 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:41:52.122594 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116326 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:41:52.122594 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116329 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:41:52.122594 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116331 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:41:52.122594 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116333 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:41:52.122594 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116336 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:41:52.122594 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116338 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:41:52.122594 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116341 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:41:52.122594 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116343 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:41:52.122594 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116346 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:41:52.122594 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116348 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:41:52.122594 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116350 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:41:52.122594 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116353 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:41:52.122594 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116355 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:41:52.122594 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116358 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:41:52.122594 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116360 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:41:52.122594 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116363 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:41:52.122594 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116365 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:41:52.122594 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116368 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:41:52.123097 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116370 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:41:52.123097 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116373 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:41:52.123097 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116375 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:41:52.123097 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116377 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:41:52.123097 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116380 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:41:52.123097 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116382 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:41:52.123097 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116385 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:41:52.123097 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116387 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:41:52.123097 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116389 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:41:52.123097 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116392 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:41:52.123097 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116395 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:41:52.123097 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116398 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:41:52.123097 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116401 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:41:52.123097 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116404 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:41:52.123097 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116407 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:41:52.123097 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116409 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:41:52.123097 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116411 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:41:52.123097 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116414 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:41:52.123097 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:52.116418 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:41:52.123584 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.116422 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 15:41:52.123584 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.116543 2575 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 15:41:52.123584 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.118581 2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 15:41:52.123584 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.120007 2575 server.go:1019] "Starting client certificate rotation" Jul 07 15:41:52.123584 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.120144 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 15:41:52.123584 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.120177 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 15:41:52.123584 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.122626 2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 15:41:52.133336 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.133312 2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 15:41:52.145735 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.145710 2575 log.go:25] "Validated CRI v1 runtime API" Jul 07 15:41:52.147868 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.147842 2575 log.go:25] "Validated CRI v1 image API" Jul 07 15:41:52.149314 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.149295 2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 15:41:52.149508 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.149492 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 15:41:52.152490 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.152469 2575 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 c78616ee-71bc-44fb-a516-5c7a8c0bed06:/dev/nvme0n1p3 dc3ca917-38af-4303-a686-4eccf2852806:/dev/nvme0n1p4] Jul 07 15:41:52.152568 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.152490 2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 15:41:52.157708 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.157566 2575 manager.go:216] Machine: {Timestamp:2026-07-07 15:41:52.156208658 +0000 UTC m=+0.359322489 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3107124 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2930fadb71e8972a3f74ba7b83fc4c SystemUUID:ec2930fa-db71-e897-2a3f-74ba7b83fc4c BootID:cd58abd9-5bd1-49f8-8065-b0e5eeedf214 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:42:df:84:c0:a3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:42:df:84:c0:a3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2e:57:12:4d:8c:74 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 15:41:52.157708 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.157706 2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 15:41:52.157815 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.157796 2575 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 15:41:52.160505 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.160483 2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 15:41:52.160646 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.160507 2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-221.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 15:41:52.161176 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.161165 2575 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 15:41:52.161215 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.161177 2575 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 15:41:52.161215 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.161189 2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 15:41:52.161777 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.161766 2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 15:41:52.162951 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.162941 2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 15:41:52.166066 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.166050 2575 state_mem.go:36] "Initialized new in-memory state store" Jul 07 15:41:52.166212 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.166200 2575 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 15:41:52.166271 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.166241 2575 kubelet.go:486] "Attempting to sync node with API server" Jul 07 15:41:52.166271 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.166261 2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 15:41:52.166353 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.166278 2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 15:41:52.166353 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.166294 2575 kubelet.go:398] "Adding apiserver pod source" Jul 07 15:41:52.166353 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.166307 2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 15:41:52.166712 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.166699 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 15:41:52.166783 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.166727 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 15:41:52.169421 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.169403 2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 15:41:52.171114 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.171098 2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 15:41:52.171201 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.171136 2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 15:41:52.172148 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.172134 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 15:41:52.172214 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.172156 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 15:41:52.172214 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.172165 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 15:41:52.172214 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.172173 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 15:41:52.172214 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.172186 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 15:41:52.172214 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.172194 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 15:41:52.172214 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.172206 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 15:41:52.172214 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.172214 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 15:41:52.172431 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.172225 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 15:41:52.172431 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.172234 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 15:41:52.172431 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.172246 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 15:41:52.172431 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.172254 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 15:41:52.172937 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.172925 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 15:41:52.172997 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.172940 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 15:41:52.176261 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.176239 2575 server.go:1268] "Started kubelet" Jul 07 15:41:52.176375 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.176348 2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 15:41:52.176469 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.176404 2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 15:41:52.180085 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.180063 2575 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 15:41:52.181096 ip-10-0-131-221 systemd[1]: Started Kubernetes Kubelet. Jul 07 15:41:52.181675 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.181655 2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 15:41:52.183615 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.183600 2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-221.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 15:41:52.183855 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:52.183823 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-221.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 15:41:52.184982 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:52.184933 2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 15:41:52.185575 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:52.185394 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 15:41:52.186009 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.185988 2575 server.go:310] "Adding debug handlers to kubelet server" Jul 07 15:41:52.189163 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:52.188017 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-221.ec2.internal.18c00c3dd6df3c52 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-221.ec2.internal,UID:ip-10-0-131-221.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-221.ec2.internal,},FirstTimestamp:2026-07-07 15:41:52.176208978 +0000 UTC m=+0.379322809,LastTimestamp:2026-07-07 15:41:52.176208978 +0000 UTC m=+0.379322809,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-221.ec2.internal,}" Jul 07 15:41:52.190355 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.190335 2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 15:41:52.190435 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.190338 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 15:41:52.190507 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:52.190489 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-221.ec2.internal\" not found" Jul 07 15:41:52.191101 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.191075 2575 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 15:41:52.191239 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.191223 2575 factory.go:55] Registering systemd factory Jul 07 15:41:52.191322 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.191244 2575 factory.go:223] Registration of the systemd container factory successfully Jul 07 15:41:52.191387 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.191380 2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 15:41:52.191469 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.191454 2575 factory.go:153] Registering CRI-O factory Jul 07 15:41:52.191520 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.191471 2575 factory.go:223] Registration of the crio container factory successfully Jul 07 15:41:52.191582 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.191527 2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 15:41:52.191582 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.191553 2575 factory.go:103] Registering Raw factory Jul 07 15:41:52.191582 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.191569 2575 manager.go:1196] Started watching for new ooms in manager Jul 07 15:41:52.192042 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.192017 2575 manager.go:349] Starting recovery of all containers Jul 07 15:41:52.192836 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.192810 2575 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 15:41:52.192992 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.192979 2575 reconstruct.go:97] "Volume reconstruction finished" Jul 07 15:41:52.192992 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.192993 2575 reconciler.go:29] "Reconciler: start to sync state" Jul 07 15:41:52.194354 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:52.194329 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 15:41:52.194425 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:52.194390 2575 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-221.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 15:41:52.202660 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.202643 2575 manager.go:354] Recovery completed Jul 07 15:41:52.207753 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.207739 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:41:52.210502 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.210480 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-221.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:41:52.210579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.210521 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-221.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:41:52.210579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.210536 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-221.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:41:52.211107 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.211091 2575 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 15:41:52.211107 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.211106 2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 15:41:52.211266 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.211130 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:41:52.211266 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.211157 2575 state_mem.go:36] "Initialized new in-memory state store" Jul 07 15:41:52.211856 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.211842 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-221.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:41:52.211927 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.211867 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-221.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:41:52.211927 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.211881 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-221.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:41:52.212375 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:52.212302 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-221.ec2.internal.18c00c3dd8ea8528 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-221.ec2.internal,UID:ip-10-0-131-221.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-221.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-221.ec2.internal,},FirstTimestamp:2026-07-07 15:41:52.210502952 +0000 UTC m=+0.413616788,LastTimestamp:2026-07-07 15:41:52.210502952 +0000 UTC m=+0.413616788,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-221.ec2.internal,}" Jul 07 15:41:52.212563 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.212549 2575 policy_none.go:49] "None policy: Start" Jul 07 15:41:52.212624 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.212569 2575 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 15:41:52.212624 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.212595 2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 15:41:52.215261 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.215241 2575 policy_none.go:47] "Start" Jul 07 15:41:52.219142 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.218995 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5m64b" Jul 07 15:41:52.227406 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:52.227259 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-221.ec2.internal.18c00c3dd8eae969 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-221.ec2.internal,UID:ip-10-0-131-221.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-221.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-221.ec2.internal,},FirstTimestamp:2026-07-07 15:41:52.210528617 +0000 UTC m=+0.413642453,LastTimestamp:2026-07-07 15:41:52.210528617 +0000 UTC m=+0.413642453,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-221.ec2.internal,}" Jul 07 15:41:52.231773 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.231751 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5m64b" Jul 07 15:41:52.239624 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.239610 2575 manager.go:337] "Starting Device Plugin manager" Jul 07 15:41:52.239710 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:52.239646 2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 15:41:52.239710 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.239658 2575 server.go:87] "Starting device plugin registration server" Jul 07 15:41:52.239906 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.239894 2575 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 15:41:52.239966 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.239908 2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 15:41:52.240193 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.240093 2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 15:41:52.240193 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.240168 2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 15:41:52.240193 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.240175 2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 15:41:52.241829 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:52.241595 2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 15:41:52.241829 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:52.241652 2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-221.ec2.internal\" not found" Jul 07 15:41:52.320194 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.320148 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 15:41:52.321402 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.321379 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 15:41:52.321513 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.321430 2575 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 15:41:52.321513 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.321457 2575 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 15:41:52.321513 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:52.321503 2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 15:41:52.323797 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.323776 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:41:52.340541 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.340520 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:41:52.341531 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.341514 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-221.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:41:52.341600 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.341547 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-221.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:41:52.341600 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.341562 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-221.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:41:52.341600 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.341589 2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-221.ec2.internal" Jul 07 15:41:52.352949 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.352907 2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-221.ec2.internal" Jul 07 15:41:52.352949 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:52.352930 2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-221.ec2.internal\": node \"ip-10-0-131-221.ec2.internal\" not found" Jul 07 15:41:52.392997 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:52.392972 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-221.ec2.internal\" not found" Jul 07 15:41:52.422079 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.422011 2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-131-221.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-221.ec2.internal"] Jul 07 15:41:52.422204 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.422120 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:41:52.423650 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.423630 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-221.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:41:52.423752 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.423664 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-221.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:41:52.423752 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.423677 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-221.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:41:52.424986 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.424974 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:41:52.425132 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.425101 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-221.ec2.internal" Jul 07 15:41:52.425172 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.425148 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:41:52.425696 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.425677 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-221.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:41:52.425796 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.425711 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-221.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:41:52.425796 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.425722 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-221.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:41:52.425796 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.425680 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-221.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:41:52.425796 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.425784 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-221.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:41:52.425796 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.425798 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-221.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:41:52.427009 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.426992 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-221.ec2.internal" Jul 07 15:41:52.427094 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.427018 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:41:52.427873 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.427844 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-221.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:41:52.427964 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.427877 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-221.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:41:52.427964 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.427892 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-221.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:41:52.448575 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:52.448556 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-221.ec2.internal\" not found" node="ip-10-0-131-221.ec2.internal" Jul 07 15:41:52.451908 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:52.451893 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-221.ec2.internal\" not found" node="ip-10-0-131-221.ec2.internal" Jul 07 15:41:52.494093 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:52.494063 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-221.ec2.internal\" not found" Jul 07 15:41:52.494223 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.494096 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/55087e15328923e7c338ebbe1c5355e7-config\") pod \"kube-apiserver-proxy-ip-10-0-131-221.ec2.internal\" (UID: \"55087e15328923e7c338ebbe1c5355e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-221.ec2.internal" Jul 07 15:41:52.494223 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.494120 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/38a982e05405693f73c35cd2825ada57-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-221.ec2.internal\" (UID: \"38a982e05405693f73c35cd2825ada57\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-221.ec2.internal" Jul 07 15:41:52.494223 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.494137 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/38a982e05405693f73c35cd2825ada57-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-221.ec2.internal\" (UID: \"38a982e05405693f73c35cd2825ada57\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-221.ec2.internal" Jul 07 15:41:52.594790 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:52.594752 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-221.ec2.internal\" not found" Jul 07 15:41:52.594936 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.594796 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/55087e15328923e7c338ebbe1c5355e7-config\") pod \"kube-apiserver-proxy-ip-10-0-131-221.ec2.internal\" (UID: \"55087e15328923e7c338ebbe1c5355e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-221.ec2.internal" Jul 07 15:41:52.594936 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.594830 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/38a982e05405693f73c35cd2825ada57-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-221.ec2.internal\" (UID: \"38a982e05405693f73c35cd2825ada57\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-221.ec2.internal" Jul 07 15:41:52.594936 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.594853 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/38a982e05405693f73c35cd2825ada57-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-221.ec2.internal\" (UID: \"38a982e05405693f73c35cd2825ada57\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-221.ec2.internal" Jul 07 15:41:52.594936 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.594906 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/55087e15328923e7c338ebbe1c5355e7-config\") pod \"kube-apiserver-proxy-ip-10-0-131-221.ec2.internal\" (UID: \"55087e15328923e7c338ebbe1c5355e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-221.ec2.internal" Jul 07 15:41:52.595103 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.594935 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/38a982e05405693f73c35cd2825ada57-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-221.ec2.internal\" (UID: \"38a982e05405693f73c35cd2825ada57\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-221.ec2.internal" Jul 07 15:41:52.595103 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.594906 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/38a982e05405693f73c35cd2825ada57-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-221.ec2.internal\" (UID: \"38a982e05405693f73c35cd2825ada57\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-221.ec2.internal" Jul 07 15:41:52.695552 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:52.695480 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-221.ec2.internal\" not found" Jul 07 15:41:52.751725 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.751696 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-221.ec2.internal" Jul 07 15:41:52.755752 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:52.755523 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-221.ec2.internal" Jul 07 15:41:52.796420 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:52.796392 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-221.ec2.internal\" not found" Jul 07 15:41:54.004636 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:54.004595 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-221.ec2.internal\" not found" Jul 07 15:41:54.105128 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:54.105039 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-221.ec2.internal\" not found" Jul 07 15:41:54.205454 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:54.205423 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-221.ec2.internal\" not found" Jul 07 15:41:54.227614 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:54.227585 2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 15:41:54.227775 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:54.227730 2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 15:41:54.305573 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:54.305543 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-221.ec2.internal\" not found" Jul 07 15:41:54.341801 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:54.341766 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 15:36:53 +0000 UTC" deadline="2028-03-29 05:27:06.272942074 +0000 UTC" Jul 07 15:41:54.341801 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:54.341799 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15133h45m11.931147392s" Jul 07 15:41:54.406380 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:54.406342 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-221.ec2.internal\" not found" Jul 07 15:41:54.443787 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:54.443760 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:41:54.444920 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:54.444899 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-221.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:41:54.445053 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:54.444930 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-221.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:41:54.445053 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:54.444940 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-221.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:41:54.465003 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:54.464976 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55087e15328923e7c338ebbe1c5355e7.slice/crio-dcf7359d5687b9c303c6349db2549dcc7fe451c334d4013db3e2941d3dc90628 WatchSource:0}: Error finding container dcf7359d5687b9c303c6349db2549dcc7fe451c334d4013db3e2941d3dc90628: Status 404 returned error can't find the container with id dcf7359d5687b9c303c6349db2549dcc7fe451c334d4013db3e2941d3dc90628 Jul 07 15:41:54.469577 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:54.469552 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 15:41:54.507190 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:54.507162 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-221.ec2.internal\" not found" Jul 07 15:41:54.607592 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:54.607560 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-221.ec2.internal\" not found" Jul 07 15:41:54.671050 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:54.671014 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38a982e05405693f73c35cd2825ada57.slice/crio-25cb3971dfe812665ec128b64fba6fb009e0e2cb5792c6e4932cf115ab8eafda WatchSource:0}: Error finding container 25cb3971dfe812665ec128b64fba6fb009e0e2cb5792c6e4932cf115ab8eafda: Status 404 returned error can't find the container with id 25cb3971dfe812665ec128b64fba6fb009e0e2cb5792c6e4932cf115ab8eafda Jul 07 15:41:54.708455 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:54.708419 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-221.ec2.internal\" not found" Jul 07 15:41:54.809050 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:54.809010 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-221.ec2.internal\" not found" Jul 07 15:41:54.828198 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:54.828168 2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:41:54.834813 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:54.834787 2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:41:54.898533 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:54.898501 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-221.ec2.internal" Jul 07 15:41:54.903182 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:54.903153 2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:41:54.909946 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:54.909925 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 15:41:54.910953 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:54.910934 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-221.ec2.internal" Jul 07 15:41:54.949101 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:54.949020 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 15:41:55.275124 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.275065 2575 apiserver.go:52] "Watching apiserver" Jul 07 15:41:55.285050 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.285023 2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 15:41:55.286246 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.286224 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-wtwmv","openshift-dns/node-resolver-mpvp5","openshift-image-registry/node-ca-9ng9d","openshift-multus/multus-ktmcm","openshift-multus/network-metrics-daemon-6p7tj","openshift-network-diagnostics/network-check-target-fwrrt","openshift-network-operator/iptables-alerter-qdncm","kube-system/kube-apiserver-proxy-ip-10-0-131-221.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5","openshift-cluster-node-tuning-operator/tuned-q9prp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-221.ec2.internal","openshift-multus/multus-additional-cni-plugins-g42hc","openshift-ovn-kubernetes/ovnkube-node-4d86s","kube-system/global-pull-secret-syncer-nkjmf"] Jul 07 15:41:55.287726 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.287708 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qdncm" Jul 07 15:41:55.288951 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.288931 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.290118 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.290100 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9ng9d" Jul 07 15:41:55.290536 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.290516 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 15:41:55.290654 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.290536 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 15:41:55.290654 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.290583 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-d74zx\"" Jul 07 15:41:55.291067 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.291054 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:41:55.291228 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.291208 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.291328 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.291308 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 15:41:55.291830 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.291813 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 15:41:55.292034 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.292020 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 15:41:55.292637 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.292620 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:41:55.292720 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:55.292698 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6p7tj" podUID="39adfd20-ec8f-4114-ae3e-f6abc9c340f3" Jul 07 15:41:55.293394 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.293358 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 15:41:55.293470 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.293403 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 15:41:55.293470 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.293429 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 15:41:55.293686 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.293668 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 15:41:55.293780 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.293770 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hjx4w\"" Jul 07 15:41:55.293840 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.293823 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 15:41:55.294261 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.294121 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:41:55.294261 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:55.294225 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwrrt" podUID="1b09eb17-9cb5-42f8-9633-61378ad652d3" Jul 07 15:41:55.294944 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.294769 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 15:41:55.295042 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.294958 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 15:41:55.295878 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.295112 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 15:41:55.295878 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.295284 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rmt8t\"" Jul 07 15:41:55.295878 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.295480 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 15:41:55.295878 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.295619 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 15:41:55.296259 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.296237 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" Jul 07 15:41:55.296316 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.296274 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-2p89f\"" Jul 07 15:41:55.298609 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.297155 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wtwmv" Jul 07 15:41:55.298609 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.298427 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 15:41:55.300063 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.299434 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mpvp5" Jul 07 15:41:55.300063 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.299715 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 15:41:55.300063 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.299792 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 15:41:55.300063 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.299799 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 15:41:55.300063 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.299823 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xs6gk\"" Jul 07 15:41:55.300063 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.299967 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2n5p4\"" Jul 07 15:41:55.300303 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.300237 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 15:41:55.300844 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.300824 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 15:41:55.301998 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.301979 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 15:41:55.302115 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.302097 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.302966 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.302949 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 15:41:55.303063 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.303047 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pmqks\"" Jul 07 15:41:55.303619 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.303601 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:41:55.303701 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:55.303681 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkjmf" podUID="e207ff37-6b1c-4da4-8327-54488837f74d" Jul 07 15:41:55.304930 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.304909 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.305021 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.304950 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mcb7s\"" Jul 07 15:41:55.305021 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.304999 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:41:55.305133 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.305088 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 15:41:55.307381 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.307363 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xt7wk\"" Jul 07 15:41:55.307961 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.307943 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 15:41:55.308219 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.308206 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 15:41:55.311566 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.311544 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e082571a-1e86-45e9-8fd4-3d07a6fe16d8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xmhj5\" (UID: \"e082571a-1e86-45e9-8fd4-3d07a6fe16d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" Jul 07 15:41:55.311665 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.311577 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-var-lib-openvswitch\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.311665 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.311630 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-etc-openvswitch\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.311665 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.311653 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-host-cni-bin\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.311823 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.311671 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e082571a-1e86-45e9-8fd4-3d07a6fe16d8-socket-dir\") pod \"aws-ebs-csi-driver-node-xmhj5\" (UID: \"e082571a-1e86-45e9-8fd4-3d07a6fe16d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" Jul 07 15:41:55.311823 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.311695 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56652ce6-f2fe-44ee-a263-72343b20fafe-tmp-dir\") pod \"node-resolver-mpvp5\" (UID: \"56652ce6-f2fe-44ee-a263-72343b20fafe\") " pod="openshift-dns/node-resolver-mpvp5" Jul 07 15:41:55.311823 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.311724 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-run\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.311823 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.311746 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-host\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.311823 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.311766 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-run-openvswitch\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.311823 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.311791 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0efee2cb-622f-4f59-a8a4-1eecfc38ead4-host\") pod \"node-ca-9ng9d\" (UID: \"0efee2cb-622f-4f59-a8a4-1eecfc38ead4\") " pod="openshift-image-registry/node-ca-9ng9d" Jul 07 15:41:55.312030 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.311835 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-host-var-lib-cni-multus\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.312030 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.311857 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1618aa2c-2141-4cc4-9ba1-12228efbece3-multus-daemon-config\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.312030 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.311905 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs\") pod \"network-metrics-daemon-6p7tj\" (UID: \"39adfd20-ec8f-4114-ae3e-f6abc9c340f3\") " pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:41:55.312030 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.311921 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-etc-kubernetes\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.312030 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.311955 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-etc-systemd\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.312030 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.311979 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-lib-modules\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.312030 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312007 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-host-kubelet\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.312310 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312047 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d2bba107-5e45-467a-8029-9a710c0cf2f4-host-slash\") pod \"iptables-alerter-qdncm\" (UID: \"d2bba107-5e45-467a-8029-9a710c0cf2f4\") " pod="openshift-network-operator/iptables-alerter-qdncm" Jul 07 15:41:55.312310 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312076 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-etc-modprobe-d\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.312310 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312134 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e082571a-1e86-45e9-8fd4-3d07a6fe16d8-device-dir\") pod \"aws-ebs-csi-driver-node-xmhj5\" (UID: \"e082571a-1e86-45e9-8fd4-3d07a6fe16d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" Jul 07 15:41:55.312310 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312158 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krxx8\" (UniqueName: \"kubernetes.io/projected/e082571a-1e86-45e9-8fd4-3d07a6fe16d8-kube-api-access-krxx8\") pod \"aws-ebs-csi-driver-node-xmhj5\" (UID: \"e082571a-1e86-45e9-8fd4-3d07a6fe16d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" Jul 07 15:41:55.312310 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312175 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fdh4\" (UniqueName: \"kubernetes.io/projected/1618aa2c-2141-4cc4-9ba1-12228efbece3-kube-api-access-2fdh4\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.312310 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312209 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b487\" (UniqueName: \"kubernetes.io/projected/b6fd83b0-319e-43d3-a926-256524660c79-kube-api-access-2b487\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.312310 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312232 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-cnibin\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.312310 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312248 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9r42\" (UniqueName: \"kubernetes.io/projected/56652ce6-f2fe-44ee-a263-72343b20fafe-kube-api-access-t9r42\") pod \"node-resolver-mpvp5\" (UID: \"56652ce6-f2fe-44ee-a263-72343b20fafe\") " pod="openshift-dns/node-resolver-mpvp5" Jul 07 15:41:55.312310 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312261 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-sys\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.312310 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312275 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-systemd-units\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.312310 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312296 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-os-release\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.312902 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312318 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-multus-socket-dir-parent\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.312902 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312357 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-host-run-k8s-cni-cncf-io\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.312902 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312380 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-host-var-lib-cni-bin\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.312902 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312402 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/56652ce6-f2fe-44ee-a263-72343b20fafe-hosts-file\") pod \"node-resolver-mpvp5\" (UID: \"56652ce6-f2fe-44ee-a263-72343b20fafe\") " pod="openshift-dns/node-resolver-mpvp5" Jul 07 15:41:55.312902 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312446 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-node-log\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.312902 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312478 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d2bba107-5e45-467a-8029-9a710c0cf2f4-iptables-alerter-script\") pod \"iptables-alerter-qdncm\" (UID: \"d2bba107-5e45-467a-8029-9a710c0cf2f4\") " pod="openshift-network-operator/iptables-alerter-qdncm" Jul 07 15:41:55.312902 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312500 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-multus-conf-dir\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.312902 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312566 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4500e026-5b93-4ce5-94ed-024a6432ab99-konnectivity-ca\") pod \"konnectivity-agent-wtwmv\" (UID: \"4500e026-5b93-4ce5-94ed-024a6432ab99\") " pod="kube-system/konnectivity-agent-wtwmv" Jul 07 15:41:55.312902 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312615 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ssr9\" (UniqueName: \"kubernetes.io/projected/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-kube-api-access-4ssr9\") pod \"network-metrics-daemon-6p7tj\" (UID: \"39adfd20-ec8f-4114-ae3e-f6abc9c340f3\") " pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:41:55.312902 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312641 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-run-systemd\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.312902 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312666 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kmsr\" (UniqueName: \"kubernetes.io/projected/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-kube-api-access-6kmsr\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.312902 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312722 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-system-cni-dir\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.312902 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312748 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-host-run-netns\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.312902 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312774 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-host-var-lib-kubelet\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.312902 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312809 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e082571a-1e86-45e9-8fd4-3d07a6fe16d8-registration-dir\") pod \"aws-ebs-csi-driver-node-xmhj5\" (UID: \"e082571a-1e86-45e9-8fd4-3d07a6fe16d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" Jul 07 15:41:55.312902 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312832 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-log-socket\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.313574 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312853 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-host-cni-netd\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.313574 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312889 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-ovn-node-metrics-cert\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.313574 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312915 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwh5j\" (UniqueName: \"kubernetes.io/projected/1b09eb17-9cb5-42f8-9633-61378ad652d3-kube-api-access-qwh5j\") pod \"network-check-target-fwrrt\" (UID: \"1b09eb17-9cb5-42f8-9633-61378ad652d3\") " pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:41:55.313574 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312937 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-hostroot\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.313574 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.312982 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0efee2cb-622f-4f59-a8a4-1eecfc38ead4-serviceca\") pod \"node-ca-9ng9d\" (UID: \"0efee2cb-622f-4f59-a8a4-1eecfc38ead4\") " pod="openshift-image-registry/node-ca-9ng9d" Jul 07 15:41:55.313574 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.313014 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-var-lib-kubelet\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.313574 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.313040 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-host-run-ovn-kubernetes\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.313574 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.313069 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.313574 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.313094 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd9db\" (UniqueName: \"kubernetes.io/projected/d2bba107-5e45-467a-8029-9a710c0cf2f4-kube-api-access-nd9db\") pod \"iptables-alerter-qdncm\" (UID: \"d2bba107-5e45-467a-8029-9a710c0cf2f4\") " pod="openshift-network-operator/iptables-alerter-qdncm" Jul 07 15:41:55.313574 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.313118 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-ovnkube-config\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.313574 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.313142 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-etc-sysconfig\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.313574 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.313167 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-etc-sysctl-conf\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.313574 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.313205 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6fd83b0-319e-43d3-a926-256524660c79-tmp\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.313574 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.313242 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e082571a-1e86-45e9-8fd4-3d07a6fe16d8-etc-selinux\") pod \"aws-ebs-csi-driver-node-xmhj5\" (UID: \"e082571a-1e86-45e9-8fd4-3d07a6fe16d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" Jul 07 15:41:55.313574 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.313268 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e082571a-1e86-45e9-8fd4-3d07a6fe16d8-sys-fs\") pod \"aws-ebs-csi-driver-node-xmhj5\" (UID: \"e082571a-1e86-45e9-8fd4-3d07a6fe16d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" Jul 07 15:41:55.313574 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.313288 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-run-ovn\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.314309 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.313310 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-env-overrides\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.314309 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.313330 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-etc-sysctl-d\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.314309 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.313352 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-etc-kubernetes\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.314309 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.313374 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b6fd83b0-319e-43d3-a926-256524660c79-etc-tuned\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.314309 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.313395 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-host-slash\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.314309 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.313418 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-host-run-netns\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.314309 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.313442 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4500e026-5b93-4ce5-94ed-024a6432ab99-agent-certs\") pod \"konnectivity-agent-wtwmv\" (UID: \"4500e026-5b93-4ce5-94ed-024a6432ab99\") " pod="kube-system/konnectivity-agent-wtwmv" Jul 07 15:41:55.314309 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.313464 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-multus-cni-dir\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.314309 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.313489 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 15:41:55.314309 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.313496 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1618aa2c-2141-4cc4-9ba1-12228efbece3-cni-binary-copy\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.314309 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.313521 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-ovnkube-script-lib\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.314309 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.313545 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blcmt\" (UniqueName: \"kubernetes.io/projected/0efee2cb-622f-4f59-a8a4-1eecfc38ead4-kube-api-access-blcmt\") pod \"node-ca-9ng9d\" (UID: \"0efee2cb-622f-4f59-a8a4-1eecfc38ead4\") " pod="openshift-image-registry/node-ca-9ng9d" Jul 07 15:41:55.314309 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.313577 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-host-run-multus-certs\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.351710 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.351689 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mxmw4" Jul 07 15:41:55.360969 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.360944 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mxmw4" Jul 07 15:41:55.400732 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.400699 2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 15:41:55.414002 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.413965 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/56652ce6-f2fe-44ee-a263-72343b20fafe-hosts-file\") pod \"node-resolver-mpvp5\" (UID: \"56652ce6-f2fe-44ee-a263-72343b20fafe\") " pod="openshift-dns/node-resolver-mpvp5" Jul 07 15:41:55.414187 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414011 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-node-log\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.414187 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414036 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d2bba107-5e45-467a-8029-9a710c0cf2f4-iptables-alerter-script\") pod \"iptables-alerter-qdncm\" (UID: \"d2bba107-5e45-467a-8029-9a710c0cf2f4\") " pod="openshift-network-operator/iptables-alerter-qdncm" Jul 07 15:41:55.414187 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414059 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-multus-conf-dir\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.414187 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414085 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd01334a-48d5-49a8-b01e-9f2956b5bf40-cnibin\") pod \"multus-additional-cni-plugins-g42hc\" (UID: \"dd01334a-48d5-49a8-b01e-9f2956b5bf40\") " pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.414187 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414096 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-node-log\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.414187 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414110 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4500e026-5b93-4ce5-94ed-024a6432ab99-konnectivity-ca\") pod \"konnectivity-agent-wtwmv\" (UID: \"4500e026-5b93-4ce5-94ed-024a6432ab99\") " pod="kube-system/konnectivity-agent-wtwmv" Jul 07 15:41:55.414187 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414162 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ssr9\" (UniqueName: \"kubernetes.io/projected/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-kube-api-access-4ssr9\") pod \"network-metrics-daemon-6p7tj\" (UID: \"39adfd20-ec8f-4114-ae3e-f6abc9c340f3\") " pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:41:55.414569 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414189 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-run-systemd\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.414569 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414227 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kmsr\" (UniqueName: \"kubernetes.io/projected/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-kube-api-access-6kmsr\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.414569 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414089 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/56652ce6-f2fe-44ee-a263-72343b20fafe-hosts-file\") pod \"node-resolver-mpvp5\" (UID: \"56652ce6-f2fe-44ee-a263-72343b20fafe\") " pod="openshift-dns/node-resolver-mpvp5" Jul 07 15:41:55.414569 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414253 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-system-cni-dir\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.414569 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414279 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-multus-conf-dir\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.414569 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414302 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-system-cni-dir\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.414569 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414321 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-run-systemd\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.414569 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414328 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-host-run-netns\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.414569 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414357 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-host-var-lib-kubelet\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.414569 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414385 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e082571a-1e86-45e9-8fd4-3d07a6fe16d8-registration-dir\") pod \"aws-ebs-csi-driver-node-xmhj5\" (UID: \"e082571a-1e86-45e9-8fd4-3d07a6fe16d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" Jul 07 15:41:55.414569 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414410 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-log-socket\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.414569 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414431 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-host-var-lib-kubelet\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.414569 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414435 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-host-cni-netd\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.414569 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414465 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-host-run-netns\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.414569 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414477 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e082571a-1e86-45e9-8fd4-3d07a6fe16d8-registration-dir\") pod \"aws-ebs-csi-driver-node-xmhj5\" (UID: \"e082571a-1e86-45e9-8fd4-3d07a6fe16d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" Jul 07 15:41:55.414569 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414471 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-host-cni-netd\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.414569 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414490 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-ovn-node-metrics-cert\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.414569 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414507 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-log-socket\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.415457 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414531 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwh5j\" (UniqueName: \"kubernetes.io/projected/1b09eb17-9cb5-42f8-9633-61378ad652d3-kube-api-access-qwh5j\") pod \"network-check-target-fwrrt\" (UID: \"1b09eb17-9cb5-42f8-9633-61378ad652d3\") " pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:41:55.415457 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414560 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-hostroot\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.415457 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414584 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0efee2cb-622f-4f59-a8a4-1eecfc38ead4-serviceca\") pod \"node-ca-9ng9d\" (UID: \"0efee2cb-622f-4f59-a8a4-1eecfc38ead4\") " pod="openshift-image-registry/node-ca-9ng9d" Jul 07 15:41:55.415457 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414610 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd01334a-48d5-49a8-b01e-9f2956b5bf40-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g42hc\" (UID: \"dd01334a-48d5-49a8-b01e-9f2956b5bf40\") " pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.415457 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414620 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-hostroot\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.415457 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414635 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dd01334a-48d5-49a8-b01e-9f2956b5bf40-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g42hc\" (UID: \"dd01334a-48d5-49a8-b01e-9f2956b5bf40\") " pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.415457 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414663 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-var-lib-kubelet\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.415457 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414700 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d2bba107-5e45-467a-8029-9a710c0cf2f4-iptables-alerter-script\") pod \"iptables-alerter-qdncm\" (UID: \"d2bba107-5e45-467a-8029-9a710c0cf2f4\") " pod="openshift-network-operator/iptables-alerter-qdncm" Jul 07 15:41:55.415457 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414705 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-host-run-ovn-kubernetes\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.415457 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414756 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-var-lib-kubelet\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.415457 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414770 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-host-run-ovn-kubernetes\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.415457 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.414783 2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 15:41:55.415457 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415141 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0efee2cb-622f-4f59-a8a4-1eecfc38ead4-serviceca\") pod \"node-ca-9ng9d\" (UID: \"0efee2cb-622f-4f59-a8a4-1eecfc38ead4\") " pod="openshift-image-registry/node-ca-9ng9d" Jul 07 15:41:55.415457 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415184 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4500e026-5b93-4ce5-94ed-024a6432ab99-konnectivity-ca\") pod \"konnectivity-agent-wtwmv\" (UID: \"4500e026-5b93-4ce5-94ed-024a6432ab99\") " pod="kube-system/konnectivity-agent-wtwmv" Jul 07 15:41:55.415457 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415216 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.415457 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415248 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd9db\" (UniqueName: \"kubernetes.io/projected/d2bba107-5e45-467a-8029-9a710c0cf2f4-kube-api-access-nd9db\") pod \"iptables-alerter-qdncm\" (UID: \"d2bba107-5e45-467a-8029-9a710c0cf2f4\") " pod="openshift-network-operator/iptables-alerter-qdncm" Jul 07 15:41:55.415457 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415275 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-ovnkube-config\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.416314 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415299 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-etc-sysconfig\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.416314 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415304 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.416314 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415323 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-etc-sysctl-conf\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.416314 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415346 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6fd83b0-319e-43d3-a926-256524660c79-tmp\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.416314 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415380 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e082571a-1e86-45e9-8fd4-3d07a6fe16d8-etc-selinux\") pod \"aws-ebs-csi-driver-node-xmhj5\" (UID: \"e082571a-1e86-45e9-8fd4-3d07a6fe16d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" Jul 07 15:41:55.416314 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415405 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e082571a-1e86-45e9-8fd4-3d07a6fe16d8-sys-fs\") pod \"aws-ebs-csi-driver-node-xmhj5\" (UID: \"e082571a-1e86-45e9-8fd4-3d07a6fe16d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" Jul 07 15:41:55.416314 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415439 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-run-ovn\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.416314 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415461 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-env-overrides\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.416314 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415483 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-etc-sysctl-d\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.416314 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415505 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-etc-kubernetes\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.416314 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415532 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm79p\" (UniqueName: \"kubernetes.io/projected/dd01334a-48d5-49a8-b01e-9f2956b5bf40-kube-api-access-xm79p\") pod \"multus-additional-cni-plugins-g42hc\" (UID: \"dd01334a-48d5-49a8-b01e-9f2956b5bf40\") " pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.416314 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415555 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b6fd83b0-319e-43d3-a926-256524660c79-etc-tuned\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.416314 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415583 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-host-slash\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.416314 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415608 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-host-run-netns\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.416314 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415632 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4500e026-5b93-4ce5-94ed-024a6432ab99-agent-certs\") pod \"konnectivity-agent-wtwmv\" (UID: \"4500e026-5b93-4ce5-94ed-024a6432ab99\") " pod="kube-system/konnectivity-agent-wtwmv" Jul 07 15:41:55.416314 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415655 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-multus-cni-dir\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.416314 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415678 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1618aa2c-2141-4cc4-9ba1-12228efbece3-cni-binary-copy\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.417222 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415701 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-ovnkube-script-lib\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.417222 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415734 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blcmt\" (UniqueName: \"kubernetes.io/projected/0efee2cb-622f-4f59-a8a4-1eecfc38ead4-kube-api-access-blcmt\") pod \"node-ca-9ng9d\" (UID: \"0efee2cb-622f-4f59-a8a4-1eecfc38ead4\") " pod="openshift-image-registry/node-ca-9ng9d" Jul 07 15:41:55.417222 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415758 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-host-run-multus-certs\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.417222 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415781 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e082571a-1e86-45e9-8fd4-3d07a6fe16d8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xmhj5\" (UID: \"e082571a-1e86-45e9-8fd4-3d07a6fe16d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" Jul 07 15:41:55.417222 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415804 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-var-lib-openvswitch\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.417222 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415828 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-etc-openvswitch\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.417222 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415827 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-ovnkube-config\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.417222 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.415914 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-etc-sysconfig\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.417222 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.416392 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-etc-sysctl-conf\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.417222 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.416452 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-host-cni-bin\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.417222 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.416490 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e082571a-1e86-45e9-8fd4-3d07a6fe16d8-socket-dir\") pod \"aws-ebs-csi-driver-node-xmhj5\" (UID: \"e082571a-1e86-45e9-8fd4-3d07a6fe16d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" Jul 07 15:41:55.417222 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.416516 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56652ce6-f2fe-44ee-a263-72343b20fafe-tmp-dir\") pod \"node-resolver-mpvp5\" (UID: \"56652ce6-f2fe-44ee-a263-72343b20fafe\") " pod="openshift-dns/node-resolver-mpvp5" Jul 07 15:41:55.417222 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.416543 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-run\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.417222 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.416570 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-host\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.417222 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.416599 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-run-openvswitch\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.417222 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.416623 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0efee2cb-622f-4f59-a8a4-1eecfc38ead4-host\") pod \"node-ca-9ng9d\" (UID: \"0efee2cb-622f-4f59-a8a4-1eecfc38ead4\") " pod="openshift-image-registry/node-ca-9ng9d" Jul 07 15:41:55.417222 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.416654 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dd01334a-48d5-49a8-b01e-9f2956b5bf40-os-release\") pod \"multus-additional-cni-plugins-g42hc\" (UID: \"dd01334a-48d5-49a8-b01e-9f2956b5bf40\") " pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.418071 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.416692 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dd01334a-48d5-49a8-b01e-9f2956b5bf40-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g42hc\" (UID: \"dd01334a-48d5-49a8-b01e-9f2956b5bf40\") " pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.418071 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.416723 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-host-var-lib-cni-multus\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.418071 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.416752 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1618aa2c-2141-4cc4-9ba1-12228efbece3-multus-daemon-config\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.418071 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.416784 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e207ff37-6b1c-4da4-8327-54488837f74d-original-pull-secret\") pod \"global-pull-secret-syncer-nkjmf\" (UID: \"e207ff37-6b1c-4da4-8327-54488837f74d\") " pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:41:55.418071 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.416816 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd01334a-48d5-49a8-b01e-9f2956b5bf40-system-cni-dir\") pod \"multus-additional-cni-plugins-g42hc\" (UID: \"dd01334a-48d5-49a8-b01e-9f2956b5bf40\") " pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.418071 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.416842 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd01334a-48d5-49a8-b01e-9f2956b5bf40-cni-binary-copy\") pod \"multus-additional-cni-plugins-g42hc\" (UID: \"dd01334a-48d5-49a8-b01e-9f2956b5bf40\") " pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.418071 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.416940 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-host-cni-bin\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.418071 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.417050 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e082571a-1e86-45e9-8fd4-3d07a6fe16d8-socket-dir\") pod \"aws-ebs-csi-driver-node-xmhj5\" (UID: \"e082571a-1e86-45e9-8fd4-3d07a6fe16d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" Jul 07 15:41:55.418071 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.417309 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56652ce6-f2fe-44ee-a263-72343b20fafe-tmp-dir\") pod \"node-resolver-mpvp5\" (UID: \"56652ce6-f2fe-44ee-a263-72343b20fafe\") " pod="openshift-dns/node-resolver-mpvp5" Jul 07 15:41:55.418071 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.417417 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-run\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.418071 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.417475 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-host\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.418071 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.417529 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-run-openvswitch\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.418071 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.417584 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0efee2cb-622f-4f59-a8a4-1eecfc38ead4-host\") pod \"node-ca-9ng9d\" (UID: \"0efee2cb-622f-4f59-a8a4-1eecfc38ead4\") " pod="openshift-image-registry/node-ca-9ng9d" Jul 07 15:41:55.418071 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.417661 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-host-var-lib-cni-multus\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.418774 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.418381 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-host-slash\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.419421 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.419172 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-ovnkube-script-lib\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.419421 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.419249 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-var-lib-openvswitch\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.419421 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.419261 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-host-run-netns\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.419421 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.419312 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-host-run-multus-certs\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.420017 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.419598 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-etc-openvswitch\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.420017 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.419176 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e082571a-1e86-45e9-8fd4-3d07a6fe16d8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xmhj5\" (UID: \"e082571a-1e86-45e9-8fd4-3d07a6fe16d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" Jul 07 15:41:55.420167 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.420117 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1618aa2c-2141-4cc4-9ba1-12228efbece3-cni-binary-copy\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.420167 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.420154 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-multus-cni-dir\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.421439 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.420235 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-etc-kubernetes\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.421439 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.420235 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e082571a-1e86-45e9-8fd4-3d07a6fe16d8-sys-fs\") pod \"aws-ebs-csi-driver-node-xmhj5\" (UID: \"e082571a-1e86-45e9-8fd4-3d07a6fe16d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" Jul 07 15:41:55.421439 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.420314 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e082571a-1e86-45e9-8fd4-3d07a6fe16d8-etc-selinux\") pod \"aws-ebs-csi-driver-node-xmhj5\" (UID: \"e082571a-1e86-45e9-8fd4-3d07a6fe16d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" Jul 07 15:41:55.421439 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.420341 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-etc-sysctl-d\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.421439 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.420400 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-run-ovn\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.421439 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.420778 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-env-overrides\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.421439 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.420909 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs\") pod \"network-metrics-daemon-6p7tj\" (UID: \"39adfd20-ec8f-4114-ae3e-f6abc9c340f3\") " pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:41:55.421439 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.420948 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-etc-kubernetes\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.421439 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.420981 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-etc-systemd\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.421439 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421013 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-lib-modules\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.421439 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421030 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-ovn-node-metrics-cert\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.421439 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421040 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-host-kubelet\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.421439 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421074 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d2bba107-5e45-467a-8029-9a710c0cf2f4-host-slash\") pod \"iptables-alerter-qdncm\" (UID: \"d2bba107-5e45-467a-8029-9a710c0cf2f4\") " pod="openshift-network-operator/iptables-alerter-qdncm" Jul 07 15:41:55.421439 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421110 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-etc-modprobe-d\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.421439 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421142 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e082571a-1e86-45e9-8fd4-3d07a6fe16d8-device-dir\") pod \"aws-ebs-csi-driver-node-xmhj5\" (UID: \"e082571a-1e86-45e9-8fd4-3d07a6fe16d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" Jul 07 15:41:55.421439 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421171 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krxx8\" (UniqueName: \"kubernetes.io/projected/e082571a-1e86-45e9-8fd4-3d07a6fe16d8-kube-api-access-krxx8\") pod \"aws-ebs-csi-driver-node-xmhj5\" (UID: \"e082571a-1e86-45e9-8fd4-3d07a6fe16d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" Jul 07 15:41:55.421439 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421194 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-etc-systemd\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.422560 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421203 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fdh4\" (UniqueName: \"kubernetes.io/projected/1618aa2c-2141-4cc4-9ba1-12228efbece3-kube-api-access-2fdh4\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.422560 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421241 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b487\" (UniqueName: \"kubernetes.io/projected/b6fd83b0-319e-43d3-a926-256524660c79-kube-api-access-2b487\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.422560 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421275 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-cnibin\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.422560 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:55.421297 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:41:55.422560 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:55.421400 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs podName:39adfd20-ec8f-4114-ae3e-f6abc9c340f3 nodeName:}" failed. No retries permitted until 2026-07-07 15:41:55.921371854 +0000 UTC m=+3.016652938 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs") pod "network-metrics-daemon-6p7tj" (UID: "39adfd20-ec8f-4114-ae3e-f6abc9c340f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:41:55.422560 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421428 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e207ff37-6b1c-4da4-8327-54488837f74d-kubelet-config\") pod \"global-pull-secret-syncer-nkjmf\" (UID: \"e207ff37-6b1c-4da4-8327-54488837f74d\") " pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:41:55.422560 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421436 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-cnibin\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.422560 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421471 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e207ff37-6b1c-4da4-8327-54488837f74d-dbus\") pod \"global-pull-secret-syncer-nkjmf\" (UID: \"e207ff37-6b1c-4da4-8327-54488837f74d\") " pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:41:55.422560 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421501 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e082571a-1e86-45e9-8fd4-3d07a6fe16d8-device-dir\") pod \"aws-ebs-csi-driver-node-xmhj5\" (UID: \"e082571a-1e86-45e9-8fd4-3d07a6fe16d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" Jul 07 15:41:55.422560 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421511 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9r42\" (UniqueName: \"kubernetes.io/projected/56652ce6-f2fe-44ee-a263-72343b20fafe-kube-api-access-t9r42\") pod \"node-resolver-mpvp5\" (UID: \"56652ce6-f2fe-44ee-a263-72343b20fafe\") " pod="openshift-dns/node-resolver-mpvp5" Jul 07 15:41:55.422560 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421544 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-sys\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.422560 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421588 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-systemd-units\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.422560 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421618 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-os-release\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.422560 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421650 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-multus-socket-dir-parent\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.422560 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421682 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-host-run-k8s-cni-cncf-io\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.422560 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421714 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-host-var-lib-cni-bin\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.422560 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421809 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-host-var-lib-cni-bin\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.423733 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421920 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-etc-modprobe-d\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.423733 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421947 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-lib-modules\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.423733 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.421959 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-host-kubelet\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.423733 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.422022 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-etc-kubernetes\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.423733 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.422076 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-multus-socket-dir-parent\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.423733 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.422120 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-systemd-units\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.423733 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.422122 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-host-run-k8s-cni-cncf-io\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.423733 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.422176 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1618aa2c-2141-4cc4-9ba1-12228efbece3-os-release\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.423733 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.422288 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d2bba107-5e45-467a-8029-9a710c0cf2f4-host-slash\") pod \"iptables-alerter-qdncm\" (UID: \"d2bba107-5e45-467a-8029-9a710c0cf2f4\") " pod="openshift-network-operator/iptables-alerter-qdncm" Jul 07 15:41:55.423733 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.422345 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4500e026-5b93-4ce5-94ed-024a6432ab99-agent-certs\") pod \"konnectivity-agent-wtwmv\" (UID: \"4500e026-5b93-4ce5-94ed-024a6432ab99\") " pod="kube-system/konnectivity-agent-wtwmv" Jul 07 15:41:55.423733 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.422388 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6fd83b0-319e-43d3-a926-256524660c79-sys\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.423733 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.422548 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6fd83b0-319e-43d3-a926-256524660c79-tmp\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.425820 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.425785 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b6fd83b0-319e-43d3-a926-256524660c79-etc-tuned\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.425915 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.425901 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1618aa2c-2141-4cc4-9ba1-12228efbece3-multus-daemon-config\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.429243 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:55.429224 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:41:55.429328 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:55.429248 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:41:55.429328 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:55.429263 2575 projected.go:196] Error preparing data for projected volume kube-api-access-qwh5j for pod openshift-network-diagnostics/network-check-target-fwrrt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:41:55.429425 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:55.429330 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b09eb17-9cb5-42f8-9633-61378ad652d3-kube-api-access-qwh5j podName:1b09eb17-9cb5-42f8-9633-61378ad652d3 nodeName:}" failed. No retries permitted until 2026-07-07 15:41:55.929312604 +0000 UTC m=+3.024593677 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qwh5j" (UniqueName: "kubernetes.io/projected/1b09eb17-9cb5-42f8-9633-61378ad652d3-kube-api-access-qwh5j") pod "network-check-target-fwrrt" (UID: "1b09eb17-9cb5-42f8-9633-61378ad652d3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:41:55.432441 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.432373 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-blcmt\" (UniqueName: \"kubernetes.io/projected/0efee2cb-622f-4f59-a8a4-1eecfc38ead4-kube-api-access-blcmt\") pod \"node-ca-9ng9d\" (UID: \"0efee2cb-622f-4f59-a8a4-1eecfc38ead4\") " pod="openshift-image-registry/node-ca-9ng9d" Jul 07 15:41:55.432441 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.432378 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd9db\" (UniqueName: \"kubernetes.io/projected/d2bba107-5e45-467a-8029-9a710c0cf2f4-kube-api-access-nd9db\") pod \"iptables-alerter-qdncm\" (UID: \"d2bba107-5e45-467a-8029-9a710c0cf2f4\") " pod="openshift-network-operator/iptables-alerter-qdncm" Jul 07 15:41:55.435420 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.434856 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kmsr\" (UniqueName: \"kubernetes.io/projected/fbf78c29-89f3-4aad-82a7-2e8f99b6356e-kube-api-access-6kmsr\") pod \"ovnkube-node-4d86s\" (UID: \"fbf78c29-89f3-4aad-82a7-2e8f99b6356e\") " pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.435420 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.435350 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ssr9\" (UniqueName: \"kubernetes.io/projected/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-kube-api-access-4ssr9\") pod \"network-metrics-daemon-6p7tj\" (UID: \"39adfd20-ec8f-4114-ae3e-f6abc9c340f3\") " pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:41:55.435574 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.435527 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-221.ec2.internal" event={"ID":"38a982e05405693f73c35cd2825ada57","Type":"ContainerStarted","Data":"25cb3971dfe812665ec128b64fba6fb009e0e2cb5792c6e4932cf115ab8eafda"} Jul 07 15:41:55.436427 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.436401 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b487\" (UniqueName: \"kubernetes.io/projected/b6fd83b0-319e-43d3-a926-256524660c79-kube-api-access-2b487\") pod \"tuned-q9prp\" (UID: \"b6fd83b0-319e-43d3-a926-256524660c79\") " pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.437679 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.437497 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krxx8\" (UniqueName: \"kubernetes.io/projected/e082571a-1e86-45e9-8fd4-3d07a6fe16d8-kube-api-access-krxx8\") pod \"aws-ebs-csi-driver-node-xmhj5\" (UID: \"e082571a-1e86-45e9-8fd4-3d07a6fe16d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" Jul 07 15:41:55.437771 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.437737 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-221.ec2.internal" event={"ID":"55087e15328923e7c338ebbe1c5355e7","Type":"ContainerStarted","Data":"dcf7359d5687b9c303c6349db2549dcc7fe451c334d4013db3e2941d3dc90628"} Jul 07 15:41:55.438352 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.438217 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9r42\" (UniqueName: \"kubernetes.io/projected/56652ce6-f2fe-44ee-a263-72343b20fafe-kube-api-access-t9r42\") pod \"node-resolver-mpvp5\" (UID: \"56652ce6-f2fe-44ee-a263-72343b20fafe\") " pod="openshift-dns/node-resolver-mpvp5" Jul 07 15:41:55.438857 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.438749 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fdh4\" (UniqueName: \"kubernetes.io/projected/1618aa2c-2141-4cc4-9ba1-12228efbece3-kube-api-access-2fdh4\") pod \"multus-ktmcm\" (UID: \"1618aa2c-2141-4cc4-9ba1-12228efbece3\") " pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.522276 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.522239 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd01334a-48d5-49a8-b01e-9f2956b5bf40-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g42hc\" (UID: \"dd01334a-48d5-49a8-b01e-9f2956b5bf40\") " pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.522456 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.522284 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dd01334a-48d5-49a8-b01e-9f2956b5bf40-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g42hc\" (UID: \"dd01334a-48d5-49a8-b01e-9f2956b5bf40\") " pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.522456 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.522318 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm79p\" (UniqueName: \"kubernetes.io/projected/dd01334a-48d5-49a8-b01e-9f2956b5bf40-kube-api-access-xm79p\") pod \"multus-additional-cni-plugins-g42hc\" (UID: \"dd01334a-48d5-49a8-b01e-9f2956b5bf40\") " pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.522456 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.522354 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dd01334a-48d5-49a8-b01e-9f2956b5bf40-os-release\") pod \"multus-additional-cni-plugins-g42hc\" (UID: \"dd01334a-48d5-49a8-b01e-9f2956b5bf40\") " pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.522456 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.522375 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dd01334a-48d5-49a8-b01e-9f2956b5bf40-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g42hc\" (UID: \"dd01334a-48d5-49a8-b01e-9f2956b5bf40\") " pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.522456 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.522397 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e207ff37-6b1c-4da4-8327-54488837f74d-original-pull-secret\") pod \"global-pull-secret-syncer-nkjmf\" (UID: \"e207ff37-6b1c-4da4-8327-54488837f74d\") " pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:41:55.522456 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.522421 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd01334a-48d5-49a8-b01e-9f2956b5bf40-system-cni-dir\") pod \"multus-additional-cni-plugins-g42hc\" (UID: \"dd01334a-48d5-49a8-b01e-9f2956b5bf40\") " pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.522742 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.522478 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd01334a-48d5-49a8-b01e-9f2956b5bf40-cni-binary-copy\") pod \"multus-additional-cni-plugins-g42hc\" (UID: \"dd01334a-48d5-49a8-b01e-9f2956b5bf40\") " pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.522742 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.522495 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd01334a-48d5-49a8-b01e-9f2956b5bf40-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g42hc\" (UID: \"dd01334a-48d5-49a8-b01e-9f2956b5bf40\") " pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.522742 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.522523 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e207ff37-6b1c-4da4-8327-54488837f74d-kubelet-config\") pod \"global-pull-secret-syncer-nkjmf\" (UID: \"e207ff37-6b1c-4da4-8327-54488837f74d\") " pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:41:55.522742 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.522575 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e207ff37-6b1c-4da4-8327-54488837f74d-kubelet-config\") pod \"global-pull-secret-syncer-nkjmf\" (UID: \"e207ff37-6b1c-4da4-8327-54488837f74d\") " pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:41:55.522742 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.522582 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dd01334a-48d5-49a8-b01e-9f2956b5bf40-os-release\") pod \"multus-additional-cni-plugins-g42hc\" (UID: \"dd01334a-48d5-49a8-b01e-9f2956b5bf40\") " pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.522742 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.522605 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e207ff37-6b1c-4da4-8327-54488837f74d-dbus\") pod \"global-pull-secret-syncer-nkjmf\" (UID: \"e207ff37-6b1c-4da4-8327-54488837f74d\") " pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:41:55.522742 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:55.522609 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:41:55.522742 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.522669 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd01334a-48d5-49a8-b01e-9f2956b5bf40-cnibin\") pod \"multus-additional-cni-plugins-g42hc\" (UID: \"dd01334a-48d5-49a8-b01e-9f2956b5bf40\") " pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.522742 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:55.522686 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e207ff37-6b1c-4da4-8327-54488837f74d-original-pull-secret podName:e207ff37-6b1c-4da4-8327-54488837f74d nodeName:}" failed. No retries permitted until 2026-07-07 15:41:56.022668477 +0000 UTC m=+3.117949541 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e207ff37-6b1c-4da4-8327-54488837f74d-original-pull-secret") pod "global-pull-secret-syncer-nkjmf" (UID: "e207ff37-6b1c-4da4-8327-54488837f74d") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:41:55.522742 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.522706 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd01334a-48d5-49a8-b01e-9f2956b5bf40-cnibin\") pod \"multus-additional-cni-plugins-g42hc\" (UID: \"dd01334a-48d5-49a8-b01e-9f2956b5bf40\") " pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.522742 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.522731 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e207ff37-6b1c-4da4-8327-54488837f74d-dbus\") pod \"global-pull-secret-syncer-nkjmf\" (UID: \"e207ff37-6b1c-4da4-8327-54488837f74d\") " pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:41:55.523262 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.522771 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd01334a-48d5-49a8-b01e-9f2956b5bf40-system-cni-dir\") pod \"multus-additional-cni-plugins-g42hc\" (UID: \"dd01334a-48d5-49a8-b01e-9f2956b5bf40\") " pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.523262 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.523130 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dd01334a-48d5-49a8-b01e-9f2956b5bf40-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g42hc\" (UID: \"dd01334a-48d5-49a8-b01e-9f2956b5bf40\") " pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.523262 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.523165 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd01334a-48d5-49a8-b01e-9f2956b5bf40-cni-binary-copy\") pod \"multus-additional-cni-plugins-g42hc\" (UID: \"dd01334a-48d5-49a8-b01e-9f2956b5bf40\") " pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.524023 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.523994 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dd01334a-48d5-49a8-b01e-9f2956b5bf40-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g42hc\" (UID: \"dd01334a-48d5-49a8-b01e-9f2956b5bf40\") " pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.535961 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.535894 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm79p\" (UniqueName: \"kubernetes.io/projected/dd01334a-48d5-49a8-b01e-9f2956b5bf40-kube-api-access-xm79p\") pod \"multus-additional-cni-plugins-g42hc\" (UID: \"dd01334a-48d5-49a8-b01e-9f2956b5bf40\") " pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.602218 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.602178 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qdncm" Jul 07 15:41:55.608462 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.608439 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:41:55.616522 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.616498 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9ng9d" Jul 07 15:41:55.622685 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.622660 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ktmcm" Jul 07 15:41:55.628271 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.628249 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" Jul 07 15:41:55.633973 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.633954 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wtwmv" Jul 07 15:41:55.639553 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.639531 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mpvp5" Jul 07 15:41:55.650470 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.650301 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q9prp" Jul 07 15:41:55.656362 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.656297 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-g42hc" Jul 07 15:41:55.747973 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.747944 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:41:55.925486 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:55.925404 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs\") pod \"network-metrics-daemon-6p7tj\" (UID: \"39adfd20-ec8f-4114-ae3e-f6abc9c340f3\") " pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:41:55.925642 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:55.925557 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:41:55.925642 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:55.925635 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs podName:39adfd20-ec8f-4114-ae3e-f6abc9c340f3 nodeName:}" failed. No retries permitted until 2026-07-07 15:41:56.92561782 +0000 UTC m=+4.020898896 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs") pod "network-metrics-daemon-6p7tj" (UID: "39adfd20-ec8f-4114-ae3e-f6abc9c340f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:41:56.026631 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:56.026598 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwh5j\" (UniqueName: \"kubernetes.io/projected/1b09eb17-9cb5-42f8-9633-61378ad652d3-kube-api-access-qwh5j\") pod \"network-check-target-fwrrt\" (UID: \"1b09eb17-9cb5-42f8-9633-61378ad652d3\") " pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:41:56.026803 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:56.026644 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e207ff37-6b1c-4da4-8327-54488837f74d-original-pull-secret\") pod \"global-pull-secret-syncer-nkjmf\" (UID: \"e207ff37-6b1c-4da4-8327-54488837f74d\") " pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:41:56.026803 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:56.026742 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:41:56.026803 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:56.026796 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e207ff37-6b1c-4da4-8327-54488837f74d-original-pull-secret podName:e207ff37-6b1c-4da4-8327-54488837f74d nodeName:}" failed. No retries permitted until 2026-07-07 15:41:57.026782998 +0000 UTC m=+4.122064069 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e207ff37-6b1c-4da4-8327-54488837f74d-original-pull-secret") pod "global-pull-secret-syncer-nkjmf" (UID: "e207ff37-6b1c-4da4-8327-54488837f74d") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:41:56.026803 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:56.026796 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:41:56.027031 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:56.026820 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:41:56.027031 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:56.026833 2575 projected.go:196] Error preparing data for projected volume kube-api-access-qwh5j for pod openshift-network-diagnostics/network-check-target-fwrrt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:41:56.027031 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:56.026909 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b09eb17-9cb5-42f8-9633-61378ad652d3-kube-api-access-qwh5j podName:1b09eb17-9cb5-42f8-9633-61378ad652d3 nodeName:}" failed. No retries permitted until 2026-07-07 15:41:57.026891635 +0000 UTC m=+4.122172709 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qwh5j" (UniqueName: "kubernetes.io/projected/1b09eb17-9cb5-42f8-9633-61378ad652d3-kube-api-access-qwh5j") pod "network-check-target-fwrrt" (UID: "1b09eb17-9cb5-42f8-9633-61378ad652d3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:41:56.317334 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:56.317300 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4500e026_5b93_4ce5_94ed_024a6432ab99.slice/crio-cd36dbf7dbb8e79e68adb9a705a99d44073e2313331d4518e1d95733f98e35a1 WatchSource:0}: Error finding container cd36dbf7dbb8e79e68adb9a705a99d44073e2313331d4518e1d95733f98e35a1: Status 404 returned error can't find the container with id cd36dbf7dbb8e79e68adb9a705a99d44073e2313331d4518e1d95733f98e35a1 Jul 07 15:41:56.322653 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:56.322550 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbf78c29_89f3_4aad_82a7_2e8f99b6356e.slice/crio-0d32d9b924cb15cba094d7c247bf684df7cb0d8b9d40fc7aeb41812fc8bb8a99 WatchSource:0}: Error finding container 0d32d9b924cb15cba094d7c247bf684df7cb0d8b9d40fc7aeb41812fc8bb8a99: Status 404 returned error can't find the container with id 0d32d9b924cb15cba094d7c247bf684df7cb0d8b9d40fc7aeb41812fc8bb8a99 Jul 07 15:41:56.324667 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:56.324642 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1618aa2c_2141_4cc4_9ba1_12228efbece3.slice/crio-3613b047fbacefa70aaaa0866909bb2aa0960d4b77e6240299857d9f38010976 WatchSource:0}: Error finding container 3613b047fbacefa70aaaa0866909bb2aa0960d4b77e6240299857d9f38010976: Status 404 returned error can't find the container with id 3613b047fbacefa70aaaa0866909bb2aa0960d4b77e6240299857d9f38010976 Jul 07 15:41:56.325468 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:56.325445 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2bba107_5e45_467a_8029_9a710c0cf2f4.slice/crio-f98ab24c2cfe10c17e42acd2bac9e5b8b2d310e45273d3dcc7333ab1a4bc14d5 WatchSource:0}: Error finding container f98ab24c2cfe10c17e42acd2bac9e5b8b2d310e45273d3dcc7333ab1a4bc14d5: Status 404 returned error can't find the container with id f98ab24c2cfe10c17e42acd2bac9e5b8b2d310e45273d3dcc7333ab1a4bc14d5 Jul 07 15:41:56.327343 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:56.327261 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode082571a_1e86_45e9_8fd4_3d07a6fe16d8.slice/crio-58bbb2d305d1f1956be82f8afc49d602933d74225da3c8c55a8e6a19c7e146f4 WatchSource:0}: Error finding container 58bbb2d305d1f1956be82f8afc49d602933d74225da3c8c55a8e6a19c7e146f4: Status 404 returned error can't find the container with id 58bbb2d305d1f1956be82f8afc49d602933d74225da3c8c55a8e6a19c7e146f4 Jul 07 15:41:56.328554 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:56.328529 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0efee2cb_622f_4f59_a8a4_1eecfc38ead4.slice/crio-aa42fdd77d47c860b8d6aa862ed719544ddc4f65f601cd4bcb83a995f761ce29 WatchSource:0}: Error finding container aa42fdd77d47c860b8d6aa862ed719544ddc4f65f601cd4bcb83a995f761ce29: Status 404 returned error can't find the container with id aa42fdd77d47c860b8d6aa862ed719544ddc4f65f601cd4bcb83a995f761ce29 Jul 07 15:41:56.329290 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:41:56.329266 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56652ce6_f2fe_44ee_a263_72343b20fafe.slice/crio-f838c44f8158b8f3bd2d525a2bce0b27f9c0dc79e420242928bbfeea3b88acef WatchSource:0}: Error finding container f838c44f8158b8f3bd2d525a2bce0b27f9c0dc79e420242928bbfeea3b88acef: Status 404 returned error can't find the container with id f838c44f8158b8f3bd2d525a2bce0b27f9c0dc79e420242928bbfeea3b88acef Jul 07 15:41:56.364086 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:56.363895 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 15:36:55 +0000 UTC" deadline="2028-01-18 18:55:42.062606514 +0000 UTC" Jul 07 15:41:56.364189 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:56.364090 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13443h13m45.69852195s" Jul 07 15:41:56.443055 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:56.443014 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-221.ec2.internal" event={"ID":"55087e15328923e7c338ebbe1c5355e7","Type":"ContainerStarted","Data":"77504d33461513279c88cc9fea104b2f62b51f0f0018009c637c53aea7072cf3"} Jul 07 15:41:56.444514 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:56.444489 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q9prp" event={"ID":"b6fd83b0-319e-43d3-a926-256524660c79","Type":"ContainerStarted","Data":"ef8a5c972fdc46b7473dd4d41a1ac69eab1d99ddb233d26763d801304f4bbca4"} Jul 07 15:41:56.445551 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:56.445530 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g42hc" event={"ID":"dd01334a-48d5-49a8-b01e-9f2956b5bf40","Type":"ContainerStarted","Data":"c90cf9a36d63721984de5e3b9c7b2b95eaf9604e0ab02c19e2b03446e6d7fb74"} Jul 07 15:41:56.449238 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:56.449215 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mpvp5" event={"ID":"56652ce6-f2fe-44ee-a263-72343b20fafe","Type":"ContainerStarted","Data":"f838c44f8158b8f3bd2d525a2bce0b27f9c0dc79e420242928bbfeea3b88acef"} Jul 07 15:41:56.450345 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:56.450326 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9ng9d" event={"ID":"0efee2cb-622f-4f59-a8a4-1eecfc38ead4","Type":"ContainerStarted","Data":"aa42fdd77d47c860b8d6aa862ed719544ddc4f65f601cd4bcb83a995f761ce29"} Jul 07 15:41:56.452977 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:56.452958 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" event={"ID":"e082571a-1e86-45e9-8fd4-3d07a6fe16d8","Type":"ContainerStarted","Data":"58bbb2d305d1f1956be82f8afc49d602933d74225da3c8c55a8e6a19c7e146f4"} Jul 07 15:41:56.454510 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:56.454487 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qdncm" event={"ID":"d2bba107-5e45-467a-8029-9a710c0cf2f4","Type":"ContainerStarted","Data":"f98ab24c2cfe10c17e42acd2bac9e5b8b2d310e45273d3dcc7333ab1a4bc14d5"} Jul 07 15:41:56.457102 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:56.457081 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ktmcm" event={"ID":"1618aa2c-2141-4cc4-9ba1-12228efbece3","Type":"ContainerStarted","Data":"3613b047fbacefa70aaaa0866909bb2aa0960d4b77e6240299857d9f38010976"} Jul 07 15:41:56.460596 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:56.460580 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" event={"ID":"fbf78c29-89f3-4aad-82a7-2e8f99b6356e","Type":"ContainerStarted","Data":"0d32d9b924cb15cba094d7c247bf684df7cb0d8b9d40fc7aeb41812fc8bb8a99"} Jul 07 15:41:56.461621 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:56.461598 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wtwmv" event={"ID":"4500e026-5b93-4ce5-94ed-024a6432ab99","Type":"ContainerStarted","Data":"cd36dbf7dbb8e79e68adb9a705a99d44073e2313331d4518e1d95733f98e35a1"} Jul 07 15:41:56.462488 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:56.462439 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-221.ec2.internal" podStartSLOduration=2.462424059 podStartE2EDuration="2.462424059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:41:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:41:56.462326504 +0000 UTC m=+3.557607584" watchObservedRunningTime="2026-07-07 15:41:56.462424059 +0000 UTC m=+3.557705139" Jul 07 15:41:56.933772 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:56.933144 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs\") pod \"network-metrics-daemon-6p7tj\" (UID: \"39adfd20-ec8f-4114-ae3e-f6abc9c340f3\") " pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:41:56.933772 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:56.933333 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:41:56.933772 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:56.933397 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs podName:39adfd20-ec8f-4114-ae3e-f6abc9c340f3 nodeName:}" failed. No retries permitted until 2026-07-07 15:41:58.933379072 +0000 UTC m=+6.028660131 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs") pod "network-metrics-daemon-6p7tj" (UID: "39adfd20-ec8f-4114-ae3e-f6abc9c340f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:41:57.034569 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:57.034531 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e207ff37-6b1c-4da4-8327-54488837f74d-original-pull-secret\") pod \"global-pull-secret-syncer-nkjmf\" (UID: \"e207ff37-6b1c-4da4-8327-54488837f74d\") " pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:41:57.034752 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:57.034624 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwh5j\" (UniqueName: \"kubernetes.io/projected/1b09eb17-9cb5-42f8-9633-61378ad652d3-kube-api-access-qwh5j\") pod \"network-check-target-fwrrt\" (UID: \"1b09eb17-9cb5-42f8-9633-61378ad652d3\") " pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:41:57.034820 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:57.034795 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:41:57.034820 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:57.034817 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:41:57.034820 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:57.034829 2575 projected.go:196] Error preparing data for projected volume kube-api-access-qwh5j for pod openshift-network-diagnostics/network-check-target-fwrrt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:41:57.035091 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:57.034902 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b09eb17-9cb5-42f8-9633-61378ad652d3-kube-api-access-qwh5j podName:1b09eb17-9cb5-42f8-9633-61378ad652d3 nodeName:}" failed. No retries permitted until 2026-07-07 15:41:59.034884957 +0000 UTC m=+6.130166016 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qwh5j" (UniqueName: "kubernetes.io/projected/1b09eb17-9cb5-42f8-9633-61378ad652d3-kube-api-access-qwh5j") pod "network-check-target-fwrrt" (UID: "1b09eb17-9cb5-42f8-9633-61378ad652d3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:41:57.035329 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:57.035307 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:41:57.035393 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:57.035366 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e207ff37-6b1c-4da4-8327-54488837f74d-original-pull-secret podName:e207ff37-6b1c-4da4-8327-54488837f74d nodeName:}" failed. No retries permitted until 2026-07-07 15:41:59.035351254 +0000 UTC m=+6.130632326 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e207ff37-6b1c-4da4-8327-54488837f74d-original-pull-secret") pod "global-pull-secret-syncer-nkjmf" (UID: "e207ff37-6b1c-4da4-8327-54488837f74d") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:41:57.364919 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:57.364817 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 15:36:55 +0000 UTC" deadline="2027-12-24 00:47:42.344643598 +0000 UTC" Jul 07 15:41:57.364919 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:57.364854 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12825h5m44.979793645s" Jul 07 15:41:57.436125 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:57.435295 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:41:57.436125 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:57.435434 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6p7tj" podUID="39adfd20-ec8f-4114-ae3e-f6abc9c340f3" Jul 07 15:41:57.436125 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:57.435826 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:41:57.436125 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:57.435930 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwrrt" podUID="1b09eb17-9cb5-42f8-9633-61378ad652d3" Jul 07 15:41:57.436125 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:57.435999 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:41:57.436125 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:57.436071 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkjmf" podUID="e207ff37-6b1c-4da4-8327-54488837f74d" Jul 07 15:41:57.475066 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:57.474482 2575 generic.go:356] "Generic (PLEG): container finished" podID="38a982e05405693f73c35cd2825ada57" containerID="4d047cc3f727583c6dcbc0a8f7167c3e655002fbe54d9f093c0df11646b52cc1" exitCode=0 Jul 07 15:41:57.475066 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:57.474583 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-221.ec2.internal" event={"ID":"38a982e05405693f73c35cd2825ada57","Type":"ContainerDied","Data":"4d047cc3f727583c6dcbc0a8f7167c3e655002fbe54d9f093c0df11646b52cc1"} Jul 07 15:41:58.505252 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:58.504592 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-221.ec2.internal" event={"ID":"38a982e05405693f73c35cd2825ada57","Type":"ContainerStarted","Data":"ae87a01cc0819dd2980c5cf7898dc463921fd1e4f02cfa9052f49a6a112b428b"} Jul 07 15:41:58.528630 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:58.527290 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-221.ec2.internal" podStartSLOduration=4.527268814 podStartE2EDuration="4.527268814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:41:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:41:58.52616914 +0000 UTC m=+5.621450220" watchObservedRunningTime="2026-07-07 15:41:58.527268814 +0000 UTC m=+5.622549896" Jul 07 15:41:58.949143 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:58.949058 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs\") pod \"network-metrics-daemon-6p7tj\" (UID: \"39adfd20-ec8f-4114-ae3e-f6abc9c340f3\") " pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:41:58.949297 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:58.949256 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:41:58.949352 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:58.949321 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs podName:39adfd20-ec8f-4114-ae3e-f6abc9c340f3 nodeName:}" failed. No retries permitted until 2026-07-07 15:42:02.949301934 +0000 UTC m=+10.044582997 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs") pod "network-metrics-daemon-6p7tj" (UID: "39adfd20-ec8f-4114-ae3e-f6abc9c340f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:41:59.050587 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:59.049778 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwh5j\" (UniqueName: \"kubernetes.io/projected/1b09eb17-9cb5-42f8-9633-61378ad652d3-kube-api-access-qwh5j\") pod \"network-check-target-fwrrt\" (UID: \"1b09eb17-9cb5-42f8-9633-61378ad652d3\") " pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:41:59.050587 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:59.049854 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e207ff37-6b1c-4da4-8327-54488837f74d-original-pull-secret\") pod \"global-pull-secret-syncer-nkjmf\" (UID: \"e207ff37-6b1c-4da4-8327-54488837f74d\") " pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:41:59.050587 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:59.049986 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:41:59.050587 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:59.050051 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e207ff37-6b1c-4da4-8327-54488837f74d-original-pull-secret podName:e207ff37-6b1c-4da4-8327-54488837f74d nodeName:}" failed. No retries permitted until 2026-07-07 15:42:03.050033466 +0000 UTC m=+10.145314527 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e207ff37-6b1c-4da4-8327-54488837f74d-original-pull-secret") pod "global-pull-secret-syncer-nkjmf" (UID: "e207ff37-6b1c-4da4-8327-54488837f74d") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:41:59.050587 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:59.050479 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:41:59.050587 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:59.050495 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:41:59.050587 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:59.050507 2575 projected.go:196] Error preparing data for projected volume kube-api-access-qwh5j for pod openshift-network-diagnostics/network-check-target-fwrrt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:41:59.050587 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:59.050550 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b09eb17-9cb5-42f8-9633-61378ad652d3-kube-api-access-qwh5j podName:1b09eb17-9cb5-42f8-9633-61378ad652d3 nodeName:}" failed. No retries permitted until 2026-07-07 15:42:03.050535595 +0000 UTC m=+10.145816654 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qwh5j" (UniqueName: "kubernetes.io/projected/1b09eb17-9cb5-42f8-9633-61378ad652d3-kube-api-access-qwh5j") pod "network-check-target-fwrrt" (UID: "1b09eb17-9cb5-42f8-9633-61378ad652d3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:41:59.430511 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:59.430431 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:41:59.430661 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:59.430565 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwrrt" podUID="1b09eb17-9cb5-42f8-9633-61378ad652d3" Jul 07 15:41:59.430935 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:59.430914 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:41:59.431045 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:59.431024 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkjmf" podUID="e207ff37-6b1c-4da4-8327-54488837f74d" Jul 07 15:41:59.431099 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:41:59.431072 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:41:59.431186 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:41:59.431167 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6p7tj" podUID="39adfd20-ec8f-4114-ae3e-f6abc9c340f3" Jul 07 15:42:01.429719 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:01.429668 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:42:01.430312 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:01.429668 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:42:01.430312 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:01.429801 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwrrt" podUID="1b09eb17-9cb5-42f8-9633-61378ad652d3" Jul 07 15:42:01.430312 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:01.429682 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:42:01.430312 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:01.429910 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkjmf" podUID="e207ff37-6b1c-4da4-8327-54488837f74d" Jul 07 15:42:01.430312 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:01.429966 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6p7tj" podUID="39adfd20-ec8f-4114-ae3e-f6abc9c340f3" Jul 07 15:42:02.982742 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:02.982209 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs\") pod \"network-metrics-daemon-6p7tj\" (UID: \"39adfd20-ec8f-4114-ae3e-f6abc9c340f3\") " pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:42:02.982742 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:02.982344 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:42:02.982742 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:02.982407 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs podName:39adfd20-ec8f-4114-ae3e-f6abc9c340f3 nodeName:}" failed. No retries permitted until 2026-07-07 15:42:10.982387565 +0000 UTC m=+18.077668623 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs") pod "network-metrics-daemon-6p7tj" (UID: "39adfd20-ec8f-4114-ae3e-f6abc9c340f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:42:03.083524 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:03.083449 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e207ff37-6b1c-4da4-8327-54488837f74d-original-pull-secret\") pod \"global-pull-secret-syncer-nkjmf\" (UID: \"e207ff37-6b1c-4da4-8327-54488837f74d\") " pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:42:03.083694 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:03.083535 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwh5j\" (UniqueName: \"kubernetes.io/projected/1b09eb17-9cb5-42f8-9633-61378ad652d3-kube-api-access-qwh5j\") pod \"network-check-target-fwrrt\" (UID: \"1b09eb17-9cb5-42f8-9633-61378ad652d3\") " pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:42:03.083694 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:03.083576 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:42:03.083694 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:03.083633 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:42:03.083694 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:03.083641 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e207ff37-6b1c-4da4-8327-54488837f74d-original-pull-secret podName:e207ff37-6b1c-4da4-8327-54488837f74d nodeName:}" failed. No retries permitted until 2026-07-07 15:42:11.083621963 +0000 UTC m=+18.178903038 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e207ff37-6b1c-4da4-8327-54488837f74d-original-pull-secret") pod "global-pull-secret-syncer-nkjmf" (UID: "e207ff37-6b1c-4da4-8327-54488837f74d") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:42:03.083694 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:03.083648 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:42:03.083694 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:03.083661 2575 projected.go:196] Error preparing data for projected volume kube-api-access-qwh5j for pod openshift-network-diagnostics/network-check-target-fwrrt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:42:03.084040 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:03.083707 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b09eb17-9cb5-42f8-9633-61378ad652d3-kube-api-access-qwh5j podName:1b09eb17-9cb5-42f8-9633-61378ad652d3 nodeName:}" failed. No retries permitted until 2026-07-07 15:42:11.083694436 +0000 UTC m=+18.178975507 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qwh5j" (UniqueName: "kubernetes.io/projected/1b09eb17-9cb5-42f8-9633-61378ad652d3-kube-api-access-qwh5j") pod "network-check-target-fwrrt" (UID: "1b09eb17-9cb5-42f8-9633-61378ad652d3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:42:03.430594 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:03.430517 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:42:03.430757 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:03.430605 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkjmf" podUID="e207ff37-6b1c-4da4-8327-54488837f74d" Jul 07 15:42:03.430757 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:03.430611 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:42:03.430757 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:03.430696 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6p7tj" podUID="39adfd20-ec8f-4114-ae3e-f6abc9c340f3" Jul 07 15:42:03.430757 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:03.430718 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:42:03.430973 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:03.430782 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwrrt" podUID="1b09eb17-9cb5-42f8-9633-61378ad652d3" Jul 07 15:42:05.430407 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:05.430332 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:42:05.430892 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:05.430338 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:42:05.430892 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:05.430455 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkjmf" podUID="e207ff37-6b1c-4da4-8327-54488837f74d" Jul 07 15:42:05.430892 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:05.430490 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:42:05.430892 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:05.430595 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6p7tj" podUID="39adfd20-ec8f-4114-ae3e-f6abc9c340f3" Jul 07 15:42:05.430892 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:05.430659 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwrrt" podUID="1b09eb17-9cb5-42f8-9633-61378ad652d3" Jul 07 15:42:07.429900 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:07.429850 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:42:07.429900 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:07.429909 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:42:07.430375 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:07.429855 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:42:07.430375 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:07.430000 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwrrt" podUID="1b09eb17-9cb5-42f8-9633-61378ad652d3" Jul 07 15:42:07.430375 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:07.430065 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkjmf" podUID="e207ff37-6b1c-4da4-8327-54488837f74d" Jul 07 15:42:07.430375 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:07.430139 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6p7tj" podUID="39adfd20-ec8f-4114-ae3e-f6abc9c340f3" Jul 07 15:42:09.429807 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:09.429775 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:42:09.429807 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:09.429811 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:42:09.430366 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:09.429775 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:42:09.430366 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:09.429930 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwrrt" podUID="1b09eb17-9cb5-42f8-9633-61378ad652d3" Jul 07 15:42:09.430366 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:09.430031 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkjmf" podUID="e207ff37-6b1c-4da4-8327-54488837f74d" Jul 07 15:42:09.430366 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:09.430131 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6p7tj" podUID="39adfd20-ec8f-4114-ae3e-f6abc9c340f3" Jul 07 15:42:11.048294 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:11.048209 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs\") pod \"network-metrics-daemon-6p7tj\" (UID: \"39adfd20-ec8f-4114-ae3e-f6abc9c340f3\") " pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:42:11.048727 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:11.048373 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:42:11.048727 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:11.048455 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs podName:39adfd20-ec8f-4114-ae3e-f6abc9c340f3 nodeName:}" failed. No retries permitted until 2026-07-07 15:42:27.048435144 +0000 UTC m=+34.143716223 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs") pod "network-metrics-daemon-6p7tj" (UID: "39adfd20-ec8f-4114-ae3e-f6abc9c340f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:42:11.148727 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:11.148694 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwh5j\" (UniqueName: \"kubernetes.io/projected/1b09eb17-9cb5-42f8-9633-61378ad652d3-kube-api-access-qwh5j\") pod \"network-check-target-fwrrt\" (UID: \"1b09eb17-9cb5-42f8-9633-61378ad652d3\") " pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:42:11.148927 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:11.148761 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e207ff37-6b1c-4da4-8327-54488837f74d-original-pull-secret\") pod \"global-pull-secret-syncer-nkjmf\" (UID: \"e207ff37-6b1c-4da4-8327-54488837f74d\") " pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:42:11.148927 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:11.148894 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:42:11.148927 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:11.148904 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:42:11.148927 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:11.148922 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:42:11.149106 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:11.148935 2575 projected.go:196] Error preparing data for projected volume kube-api-access-qwh5j for pod openshift-network-diagnostics/network-check-target-fwrrt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:42:11.149106 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:11.148973 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e207ff37-6b1c-4da4-8327-54488837f74d-original-pull-secret podName:e207ff37-6b1c-4da4-8327-54488837f74d nodeName:}" failed. No retries permitted until 2026-07-07 15:42:27.148954273 +0000 UTC m=+34.244235353 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e207ff37-6b1c-4da4-8327-54488837f74d-original-pull-secret") pod "global-pull-secret-syncer-nkjmf" (UID: "e207ff37-6b1c-4da4-8327-54488837f74d") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:42:11.149106 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:11.148993 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b09eb17-9cb5-42f8-9633-61378ad652d3-kube-api-access-qwh5j podName:1b09eb17-9cb5-42f8-9633-61378ad652d3 nodeName:}" failed. No retries permitted until 2026-07-07 15:42:27.14898405 +0000 UTC m=+34.244265109 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qwh5j" (UniqueName: "kubernetes.io/projected/1b09eb17-9cb5-42f8-9633-61378ad652d3-kube-api-access-qwh5j") pod "network-check-target-fwrrt" (UID: "1b09eb17-9cb5-42f8-9633-61378ad652d3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:42:11.429637 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:11.429600 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:42:11.429817 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:11.429600 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:42:11.429817 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:11.429743 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6p7tj" podUID="39adfd20-ec8f-4114-ae3e-f6abc9c340f3" Jul 07 15:42:11.429817 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:11.429613 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:42:11.429997 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:11.429816 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkjmf" podUID="e207ff37-6b1c-4da4-8327-54488837f74d" Jul 07 15:42:11.429997 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:11.429889 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwrrt" podUID="1b09eb17-9cb5-42f8-9633-61378ad652d3" Jul 07 15:42:13.430699 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:13.430662 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:42:13.431204 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:13.430784 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:42:13.431204 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:13.430811 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:42:13.431204 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:13.430780 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6p7tj" podUID="39adfd20-ec8f-4114-ae3e-f6abc9c340f3" Jul 07 15:42:13.431204 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:13.430910 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwrrt" podUID="1b09eb17-9cb5-42f8-9633-61378ad652d3" Jul 07 15:42:13.431204 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:13.430981 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkjmf" podUID="e207ff37-6b1c-4da4-8327-54488837f74d" Jul 07 15:42:15.430608 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:15.430436 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:42:15.431061 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:15.430436 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:42:15.431061 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:15.430717 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkjmf" podUID="e207ff37-6b1c-4da4-8327-54488837f74d" Jul 07 15:42:15.431061 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:15.430753 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwrrt" podUID="1b09eb17-9cb5-42f8-9633-61378ad652d3" Jul 07 15:42:15.431061 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:15.430507 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:42:15.431061 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:15.430816 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6p7tj" podUID="39adfd20-ec8f-4114-ae3e-f6abc9c340f3" Jul 07 15:42:15.541413 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:15.541379 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wtwmv" event={"ID":"4500e026-5b93-4ce5-94ed-024a6432ab99","Type":"ContainerStarted","Data":"954b0641322f656cbb7a29f3abb5e352e9487e80c36de799918f51ed6d65b015"} Jul 07 15:42:15.543195 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:15.543169 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q9prp" event={"ID":"b6fd83b0-319e-43d3-a926-256524660c79","Type":"ContainerStarted","Data":"325d7e8d75f532df895065834b0faec8c75f26bcca89f0043b7f120e6d28c94b"} Jul 07 15:42:15.545067 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:15.545046 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g42hc" event={"ID":"dd01334a-48d5-49a8-b01e-9f2956b5bf40","Type":"ContainerStarted","Data":"219d821c2755acf7a1f0a712f703ef6943af3b4150e9143589318b954d42f63a"} Jul 07 15:42:15.546878 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:15.546784 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mpvp5" event={"ID":"56652ce6-f2fe-44ee-a263-72343b20fafe","Type":"ContainerStarted","Data":"f4dca74a006e13e273ff35b39e142650fec25b7cef823cf9d04286da9c43aa2d"} Jul 07 15:42:15.548521 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:15.548489 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9ng9d" event={"ID":"0efee2cb-622f-4f59-a8a4-1eecfc38ead4","Type":"ContainerStarted","Data":"18cb1f8382b765faf9291e6df250cda26038b13d924faeb5f37920a4533abc17"} Jul 07 15:42:15.550054 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:15.550033 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" event={"ID":"e082571a-1e86-45e9-8fd4-3d07a6fe16d8","Type":"ContainerStarted","Data":"75c3ad73c448110eb8d789dbfffaf8955c10528ad51364d2399cfec9d5f83036"} Jul 07 15:42:15.551398 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:15.551380 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ktmcm" event={"ID":"1618aa2c-2141-4cc4-9ba1-12228efbece3","Type":"ContainerStarted","Data":"feed348ba3adff85987e4161a4ed2f94123576e2246587ed4dc6671d8906a773"} Jul 07 15:42:15.564519 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:15.564471 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wtwmv" podStartSLOduration=3.68721874 podStartE2EDuration="22.564451749s" totalImagesPullingTime="18.877233009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:42:15.564073792 +0000 UTC m=+22.659354871" watchObservedRunningTime="2026-07-07 15:42:15.564451749 +0000 UTC m=+22.659732829" Jul 07 15:42:15.614814 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:15.614764 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mpvp5" podStartSLOduration=8.263790285 podStartE2EDuration="22.614748912s" totalImagesPullingTime="14.350958627s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:42:15.589247348 +0000 UTC m=+22.684528429" watchObservedRunningTime="2026-07-07 15:42:15.614748912 +0000 UTC m=+22.710029991" Jul 07 15:42:15.615117 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:15.615097 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-q9prp" podStartSLOduration=3.776347484 podStartE2EDuration="22.61509177s" totalImagesPullingTime="18.838744286s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:42:15.614739264 +0000 UTC m=+22.710020354" watchObservedRunningTime="2026-07-07 15:42:15.61509177 +0000 UTC m=+22.710372849" Jul 07 15:42:15.700632 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:15.700590 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9ng9d" podStartSLOduration=3.8566576169999998 podStartE2EDuration="22.700571847s" totalImagesPullingTime="18.84391423s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:42:15.667308824 +0000 UTC m=+22.762589904" watchObservedRunningTime="2026-07-07 15:42:15.700571847 +0000 UTC m=+22.795852928" Jul 07 15:42:15.700728 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:15.700662 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ktmcm" podStartSLOduration=3.796003348 podStartE2EDuration="22.70065873s" totalImagesPullingTime="18.904655382s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:42:15.700376659 +0000 UTC m=+22.795657741" watchObservedRunningTime="2026-07-07 15:42:15.70065873 +0000 UTC m=+22.795939809" Jul 07 15:42:16.557182 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:16.557027 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4d86s_fbf78c29-89f3-4aad-82a7-2e8f99b6356e/ovn-acl-logging/0.log" Jul 07 15:42:16.557896 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:16.557498 2575 generic.go:356] "Generic (PLEG): container finished" podID="fbf78c29-89f3-4aad-82a7-2e8f99b6356e" containerID="c568cc84fb5962c6439dca984c2ea5f4c3a9b91eaefbace2ceff47f0c6f3adaa" exitCode=1 Jul 07 15:42:16.557896 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:16.557569 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" event={"ID":"fbf78c29-89f3-4aad-82a7-2e8f99b6356e","Type":"ContainerStarted","Data":"136ce676dfc74b027c9d1e6d8885604f05b085213b7fbd04b6b44a04963bfdaa"} Jul 07 15:42:16.557896 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:16.557687 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" event={"ID":"fbf78c29-89f3-4aad-82a7-2e8f99b6356e","Type":"ContainerStarted","Data":"400153abcc8802f267fe2ca7a6ea8f7e5e78055adf081c35cadb4971c9eb9e7e"} Jul 07 15:42:16.557896 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:16.557732 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" event={"ID":"fbf78c29-89f3-4aad-82a7-2e8f99b6356e","Type":"ContainerStarted","Data":"c00936d4649c57d512197568ac0cfe41d8332dbcba7a2566030bef6c5c18df49"} Jul 07 15:42:16.557896 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:16.557773 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" event={"ID":"fbf78c29-89f3-4aad-82a7-2e8f99b6356e","Type":"ContainerStarted","Data":"eb4f8bf270f6822ae5cc1b28b520007a0628220bd415f0ac5ed5a9964d3334e9"} Jul 07 15:42:16.557896 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:16.557822 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" event={"ID":"fbf78c29-89f3-4aad-82a7-2e8f99b6356e","Type":"ContainerDied","Data":"c568cc84fb5962c6439dca984c2ea5f4c3a9b91eaefbace2ceff47f0c6f3adaa"} Jul 07 15:42:16.558183 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:16.557912 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" event={"ID":"fbf78c29-89f3-4aad-82a7-2e8f99b6356e","Type":"ContainerStarted","Data":"ff16ec28cefc1347ee34fdd7723caf16742ee66baec1f4af99c717f4223f3edc"} Jul 07 15:42:16.559193 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:16.559165 2575 generic.go:356] "Generic (PLEG): container finished" podID="dd01334a-48d5-49a8-b01e-9f2956b5bf40" containerID="219d821c2755acf7a1f0a712f703ef6943af3b4150e9143589318b954d42f63a" exitCode=0 Jul 07 15:42:16.559299 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:16.559209 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g42hc" event={"ID":"dd01334a-48d5-49a8-b01e-9f2956b5bf40","Type":"ContainerDied","Data":"219d821c2755acf7a1f0a712f703ef6943af3b4150e9143589318b954d42f63a"} Jul 07 15:42:16.617055 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:16.617033 2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 15:42:16.992054 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:16.992030 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wtwmv" Jul 07 15:42:16.992567 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:16.992552 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wtwmv" Jul 07 15:42:17.366120 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:17.366011 2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T15:42:16.617049421Z","UUID":"86aae125-3eb1-4de2-a445-bc156f94846a","Handler":null,"Name":"","Endpoint":""} Jul 07 15:42:17.367608 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:17.367574 2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 15:42:17.367728 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:17.367618 2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 15:42:17.429833 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:17.429802 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:42:17.429833 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:17.429841 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:42:17.430104 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:17.429811 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:42:17.430104 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:17.429945 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwrrt" podUID="1b09eb17-9cb5-42f8-9633-61378ad652d3" Jul 07 15:42:17.430104 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:17.430031 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkjmf" podUID="e207ff37-6b1c-4da4-8327-54488837f74d" Jul 07 15:42:17.430263 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:17.430109 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6p7tj" podUID="39adfd20-ec8f-4114-ae3e-f6abc9c340f3" Jul 07 15:42:17.563252 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:17.563202 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" event={"ID":"e082571a-1e86-45e9-8fd4-3d07a6fe16d8","Type":"ContainerStarted","Data":"6fe924bf15c7a5f465fa2bf089ec8c33f1d97480a333aa8532bf7937dcc68e6c"} Jul 07 15:42:17.564917 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:17.564828 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qdncm" event={"ID":"d2bba107-5e45-467a-8029-9a710c0cf2f4","Type":"ContainerStarted","Data":"66b86272c1a14f0b1513ef6aaeb11b0e3359ebebbd7b1327ab5367eee534e168"} Jul 07 15:42:17.565165 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:17.565143 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wtwmv" Jul 07 15:42:17.565692 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:17.565673 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wtwmv" Jul 07 15:42:17.611063 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:17.611015 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qdncm" podStartSLOduration=5.762726721 podStartE2EDuration="24.610998991s" totalImagesPullingTime="18.84827227s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:42:17.58681679 +0000 UTC m=+24.682097868" watchObservedRunningTime="2026-07-07 15:42:17.610998991 +0000 UTC m=+24.706280105" Jul 07 15:42:18.570550 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:18.570270 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" event={"ID":"e082571a-1e86-45e9-8fd4-3d07a6fe16d8","Type":"ContainerStarted","Data":"b39c95ccf6c6c0c2fd606551f5016f1ca62380f0763cc3b7bf63f01dedca8531"} Jul 07 15:42:18.573951 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:18.573930 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4d86s_fbf78c29-89f3-4aad-82a7-2e8f99b6356e/ovn-acl-logging/0.log" Jul 07 15:42:18.574322 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:18.574299 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" event={"ID":"fbf78c29-89f3-4aad-82a7-2e8f99b6356e","Type":"ContainerStarted","Data":"5f5fca038a3889ab54a955ddabf52ffa327cc8664890f67e106658123f5f9bd4"} Jul 07 15:42:18.600491 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:18.600431 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmhj5" podStartSLOduration=4.693035733 podStartE2EDuration="25.600413808s" totalImagesPullingTime="20.907378075s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:41:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:42:18.600273657 +0000 UTC m=+25.695554738" watchObservedRunningTime="2026-07-07 15:42:18.600413808 +0000 UTC m=+25.695694912" Jul 07 15:42:19.430428 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:19.430399 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:42:19.430428 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:19.430422 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:42:19.430670 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:19.430399 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:42:19.430670 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:19.430495 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwrrt" podUID="1b09eb17-9cb5-42f8-9633-61378ad652d3" Jul 07 15:42:19.430670 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:19.430574 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkjmf" podUID="e207ff37-6b1c-4da4-8327-54488837f74d" Jul 07 15:42:19.430670 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:19.430653 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6p7tj" podUID="39adfd20-ec8f-4114-ae3e-f6abc9c340f3" Jul 07 15:42:21.430505 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:21.430468 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:42:21.430505 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:21.430508 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:42:21.431313 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:21.430472 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:42:21.431313 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:21.430616 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwrrt" podUID="1b09eb17-9cb5-42f8-9633-61378ad652d3" Jul 07 15:42:21.431313 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:21.431299 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkjmf" podUID="e207ff37-6b1c-4da4-8327-54488837f74d" Jul 07 15:42:21.433782 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:21.431566 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6p7tj" podUID="39adfd20-ec8f-4114-ae3e-f6abc9c340f3" Jul 07 15:42:22.587480 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:22.587303 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4d86s_fbf78c29-89f3-4aad-82a7-2e8f99b6356e/ovn-acl-logging/0.log" Jul 07 15:42:22.588095 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:22.587837 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" event={"ID":"fbf78c29-89f3-4aad-82a7-2e8f99b6356e","Type":"ContainerStarted","Data":"545ce20a6166ecab4e7a10322b8c3946d30bd29d5f43bcf4fa0e8560d06a05c7"} Jul 07 15:42:22.588189 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:22.588172 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:42:22.588293 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:22.588275 2575 scope.go:117] "RemoveContainer" containerID="c568cc84fb5962c6439dca984c2ea5f4c3a9b91eaefbace2ceff47f0c6f3adaa" Jul 07 15:42:22.590119 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:22.590094 2575 generic.go:356] "Generic (PLEG): container finished" podID="dd01334a-48d5-49a8-b01e-9f2956b5bf40" containerID="b4e4dbaa83054e8f6e46cc17bb27f32776d9e74ca2bda87068966e5126c1ae57" exitCode=0 Jul 07 15:42:22.590224 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:22.590130 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g42hc" event={"ID":"dd01334a-48d5-49a8-b01e-9f2956b5bf40","Type":"ContainerDied","Data":"b4e4dbaa83054e8f6e46cc17bb27f32776d9e74ca2bda87068966e5126c1ae57"} Jul 07 15:42:22.605495 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:22.605476 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:42:23.431085 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:23.431050 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:42:23.431268 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:23.431136 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:42:23.431268 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:23.431133 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwrrt" podUID="1b09eb17-9cb5-42f8-9633-61378ad652d3" Jul 07 15:42:23.431268 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:23.431244 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkjmf" podUID="e207ff37-6b1c-4da4-8327-54488837f74d" Jul 07 15:42:23.431268 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:23.431260 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:42:23.431419 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:23.431352 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6p7tj" podUID="39adfd20-ec8f-4114-ae3e-f6abc9c340f3" Jul 07 15:42:23.595275 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:23.595247 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4d86s_fbf78c29-89f3-4aad-82a7-2e8f99b6356e/ovn-acl-logging/0.log" Jul 07 15:42:23.595683 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:23.595662 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" event={"ID":"fbf78c29-89f3-4aad-82a7-2e8f99b6356e","Type":"ContainerStarted","Data":"0e2ae424c36449cda26775d751e9f7636722426892a1d3e73abce1387c0bae21"} Jul 07 15:42:23.595916 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:23.595891 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:42:23.595987 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:23.595967 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:42:23.613297 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:23.613275 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:42:23.639105 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:23.639060 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" podStartSLOduration=11.424882034 podStartE2EDuration="30.639044618s" totalImagesPullingTime="19.214162584s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:42:23.638617155 +0000 UTC m=+30.733898248" watchObservedRunningTime="2026-07-07 15:42:23.639044618 +0000 UTC m=+30.734325696" Jul 07 15:42:24.376485 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:24.376236 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fwrrt"] Jul 07 15:42:24.376655 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:24.376611 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:42:24.376751 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:24.376728 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwrrt" podUID="1b09eb17-9cb5-42f8-9633-61378ad652d3" Jul 07 15:42:24.379089 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:24.379063 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nkjmf"] Jul 07 15:42:24.379220 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:24.379155 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:42:24.379279 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:24.379254 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkjmf" podUID="e207ff37-6b1c-4da4-8327-54488837f74d" Jul 07 15:42:24.381569 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:24.381544 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6p7tj"] Jul 07 15:42:24.381695 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:24.381641 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:42:24.381765 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:24.381750 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6p7tj" podUID="39adfd20-ec8f-4114-ae3e-f6abc9c340f3" Jul 07 15:42:24.600262 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:24.600231 2575 generic.go:356] "Generic (PLEG): container finished" podID="dd01334a-48d5-49a8-b01e-9f2956b5bf40" containerID="df5660f3478f2cfb28428b8ccf2d7cfc1bcaae4de5a8844789006163e9db4791" exitCode=0 Jul 07 15:42:24.600713 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:24.600320 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g42hc" event={"ID":"dd01334a-48d5-49a8-b01e-9f2956b5bf40","Type":"ContainerDied","Data":"df5660f3478f2cfb28428b8ccf2d7cfc1bcaae4de5a8844789006163e9db4791"} Jul 07 15:42:26.429790 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:26.429756 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:42:26.430483 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:26.429803 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:42:26.430483 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:26.429770 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:42:26.430483 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:26.429897 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwrrt" podUID="1b09eb17-9cb5-42f8-9633-61378ad652d3" Jul 07 15:42:26.430483 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:26.429968 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6p7tj" podUID="39adfd20-ec8f-4114-ae3e-f6abc9c340f3" Jul 07 15:42:26.430483 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:26.430046 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkjmf" podUID="e207ff37-6b1c-4da4-8327-54488837f74d" Jul 07 15:42:26.607191 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:26.607161 2575 generic.go:356] "Generic (PLEG): container finished" podID="dd01334a-48d5-49a8-b01e-9f2956b5bf40" containerID="81e08a2265120b7a953c2493779adc5a5387b61247dc4a86c7d18d9b1d9dc7e7" exitCode=0 Jul 07 15:42:26.607350 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:26.607229 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g42hc" event={"ID":"dd01334a-48d5-49a8-b01e-9f2956b5bf40","Type":"ContainerDied","Data":"81e08a2265120b7a953c2493779adc5a5387b61247dc4a86c7d18d9b1d9dc7e7"} Jul 07 15:42:27.071145 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:27.071094 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs\") pod \"network-metrics-daemon-6p7tj\" (UID: \"39adfd20-ec8f-4114-ae3e-f6abc9c340f3\") " pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:42:27.071375 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:27.071266 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:42:27.071375 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:27.071342 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs podName:39adfd20-ec8f-4114-ae3e-f6abc9c340f3 nodeName:}" failed. No retries permitted until 2026-07-07 15:42:59.071323163 +0000 UTC m=+66.166604235 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs") pod "network-metrics-daemon-6p7tj" (UID: "39adfd20-ec8f-4114-ae3e-f6abc9c340f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:42:27.172212 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:27.172178 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e207ff37-6b1c-4da4-8327-54488837f74d-original-pull-secret\") pod \"global-pull-secret-syncer-nkjmf\" (UID: \"e207ff37-6b1c-4da4-8327-54488837f74d\") " pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:42:27.172377 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:27.172238 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwh5j\" (UniqueName: \"kubernetes.io/projected/1b09eb17-9cb5-42f8-9633-61378ad652d3-kube-api-access-qwh5j\") pod \"network-check-target-fwrrt\" (UID: \"1b09eb17-9cb5-42f8-9633-61378ad652d3\") " pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:42:27.172377 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:27.172353 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:42:27.172377 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:27.172375 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:42:27.172505 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:27.172388 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:42:27.172505 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:27.172397 2575 projected.go:196] Error preparing data for projected volume kube-api-access-qwh5j for pod openshift-network-diagnostics/network-check-target-fwrrt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:42:27.172505 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:27.172435 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e207ff37-6b1c-4da4-8327-54488837f74d-original-pull-secret podName:e207ff37-6b1c-4da4-8327-54488837f74d nodeName:}" failed. No retries permitted until 2026-07-07 15:42:59.172415235 +0000 UTC m=+66.267696307 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e207ff37-6b1c-4da4-8327-54488837f74d-original-pull-secret") pod "global-pull-secret-syncer-nkjmf" (UID: "e207ff37-6b1c-4da4-8327-54488837f74d") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:42:27.172505 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:27.172456 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b09eb17-9cb5-42f8-9633-61378ad652d3-kube-api-access-qwh5j podName:1b09eb17-9cb5-42f8-9633-61378ad652d3 nodeName:}" failed. No retries permitted until 2026-07-07 15:42:59.172445904 +0000 UTC m=+66.267726964 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qwh5j" (UniqueName: "kubernetes.io/projected/1b09eb17-9cb5-42f8-9633-61378ad652d3-kube-api-access-qwh5j") pod "network-check-target-fwrrt" (UID: "1b09eb17-9cb5-42f8-9633-61378ad652d3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:42:28.163635 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.163604 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-221.ec2.internal" event="NodeReady" Jul 07 15:42:28.164052 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.163753 2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 15:42:28.214424 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.214394 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-659cfff448-ggk7s"] Jul 07 15:42:28.216881 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.216839 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:28.219522 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.219500 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 15:42:28.219947 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.219926 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 15:42:28.220053 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.219925 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-d8w2m\"" Jul 07 15:42:28.220627 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.220608 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 15:42:28.224713 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.224684 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-m2fcv"] Jul 07 15:42:28.226721 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.226702 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xb8r5"] Jul 07 15:42:28.227003 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.226979 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m2fcv" Jul 07 15:42:28.228298 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.228281 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 15:42:28.229327 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.229309 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xb8r5" Jul 07 15:42:28.230132 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.229969 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 15:42:28.230132 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.229997 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 15:42:28.230132 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.230011 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kr7tb\"" Jul 07 15:42:28.230729 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.230709 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-659cfff448-ggk7s"] Jul 07 15:42:28.232156 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.231993 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 15:42:28.232744 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.232684 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 15:42:28.232836 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.232822 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-pplwf\"" Jul 07 15:42:28.233265 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.233231 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 15:42:28.247314 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.247287 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m2fcv"] Jul 07 15:42:28.248118 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.248064 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xb8r5"] Jul 07 15:42:28.380940 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.380908 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfcr9\" (UniqueName: \"kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-kube-api-access-sfcr9\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:28.381137 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.380948 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-installation-pull-secrets\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:28.381137 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.380971 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73fd3204-d75e-41e7-ae19-734ddf9d80ae-tmp-dir\") pod \"dns-default-m2fcv\" (UID: \"73fd3204-d75e-41e7-ae19-734ddf9d80ae\") " pod="openshift-dns/dns-default-m2fcv" Jul 07 15:42:28.381137 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.381072 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-certificates\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:28.381137 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.381104 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/73fd3204-d75e-41e7-ae19-734ddf9d80ae-config-volume\") pod \"dns-default-m2fcv\" (UID: \"73fd3204-d75e-41e7-ae19-734ddf9d80ae\") " pod="openshift-dns/dns-default-m2fcv" Jul 07 15:42:28.381296 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.381143 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxvd9\" (UniqueName: \"kubernetes.io/projected/19092adf-0ed9-4061-9a0a-bbc7dc75bc40-kube-api-access-xxvd9\") pod \"ingress-canary-xb8r5\" (UID: \"19092adf-0ed9-4061-9a0a-bbc7dc75bc40\") " pod="openshift-ingress-canary/ingress-canary-xb8r5" Jul 07 15:42:28.381296 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.381182 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-ca-trust-extracted\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:28.381296 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.381210 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-trusted-ca\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:28.381296 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.381251 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhwsr\" (UniqueName: \"kubernetes.io/projected/73fd3204-d75e-41e7-ae19-734ddf9d80ae-kube-api-access-lhwsr\") pod \"dns-default-m2fcv\" (UID: \"73fd3204-d75e-41e7-ae19-734ddf9d80ae\") " pod="openshift-dns/dns-default-m2fcv" Jul 07 15:42:28.381296 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.381292 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-image-registry-private-configuration\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:28.381474 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.381315 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-bound-sa-token\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:28.381474 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.381372 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73fd3204-d75e-41e7-ae19-734ddf9d80ae-metrics-tls\") pod \"dns-default-m2fcv\" (UID: \"73fd3204-d75e-41e7-ae19-734ddf9d80ae\") " pod="openshift-dns/dns-default-m2fcv" Jul 07 15:42:28.381474 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.381441 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19092adf-0ed9-4061-9a0a-bbc7dc75bc40-cert\") pod \"ingress-canary-xb8r5\" (UID: \"19092adf-0ed9-4061-9a0a-bbc7dc75bc40\") " pod="openshift-ingress-canary/ingress-canary-xb8r5" Jul 07 15:42:28.381571 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.381472 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:28.430369 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.430283 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:42:28.430369 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.430331 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:42:28.430575 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.430289 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:42:28.433409 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.433340 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 15:42:28.433409 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.433399 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 15:42:28.433571 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.433434 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 15:42:28.433571 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.433479 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-54b57\"" Jul 07 15:42:28.433571 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.433511 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 15:42:28.433788 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.433773 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-88kw6\"" Jul 07 15:42:28.482330 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.482282 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-installation-pull-secrets\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:28.482535 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.482342 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73fd3204-d75e-41e7-ae19-734ddf9d80ae-tmp-dir\") pod \"dns-default-m2fcv\" (UID: \"73fd3204-d75e-41e7-ae19-734ddf9d80ae\") " pod="openshift-dns/dns-default-m2fcv" Jul 07 15:42:28.482535 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.482384 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-certificates\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:28.482535 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.482402 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/73fd3204-d75e-41e7-ae19-734ddf9d80ae-config-volume\") pod \"dns-default-m2fcv\" (UID: \"73fd3204-d75e-41e7-ae19-734ddf9d80ae\") " pod="openshift-dns/dns-default-m2fcv" Jul 07 15:42:28.482701 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.482561 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxvd9\" (UniqueName: \"kubernetes.io/projected/19092adf-0ed9-4061-9a0a-bbc7dc75bc40-kube-api-access-xxvd9\") pod \"ingress-canary-xb8r5\" (UID: \"19092adf-0ed9-4061-9a0a-bbc7dc75bc40\") " pod="openshift-ingress-canary/ingress-canary-xb8r5" Jul 07 15:42:28.482701 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.482605 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-ca-trust-extracted\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:28.482701 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.482633 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-trusted-ca\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:28.482701 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.482654 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhwsr\" (UniqueName: \"kubernetes.io/projected/73fd3204-d75e-41e7-ae19-734ddf9d80ae-kube-api-access-lhwsr\") pod \"dns-default-m2fcv\" (UID: \"73fd3204-d75e-41e7-ae19-734ddf9d80ae\") " pod="openshift-dns/dns-default-m2fcv" Jul 07 15:42:28.482701 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.482686 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-image-registry-private-configuration\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:28.483168 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.482712 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-bound-sa-token\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:28.483168 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.482739 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73fd3204-d75e-41e7-ae19-734ddf9d80ae-metrics-tls\") pod \"dns-default-m2fcv\" (UID: \"73fd3204-d75e-41e7-ae19-734ddf9d80ae\") " pod="openshift-dns/dns-default-m2fcv" Jul 07 15:42:28.483168 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.482750 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73fd3204-d75e-41e7-ae19-734ddf9d80ae-tmp-dir\") pod \"dns-default-m2fcv\" (UID: \"73fd3204-d75e-41e7-ae19-734ddf9d80ae\") " pod="openshift-dns/dns-default-m2fcv" Jul 07 15:42:28.483168 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.482779 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19092adf-0ed9-4061-9a0a-bbc7dc75bc40-cert\") pod \"ingress-canary-xb8r5\" (UID: \"19092adf-0ed9-4061-9a0a-bbc7dc75bc40\") " pod="openshift-ingress-canary/ingress-canary-xb8r5" Jul 07 15:42:28.483168 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.483054 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/73fd3204-d75e-41e7-ae19-734ddf9d80ae-config-volume\") pod \"dns-default-m2fcv\" (UID: \"73fd3204-d75e-41e7-ae19-734ddf9d80ae\") " pod="openshift-dns/dns-default-m2fcv" Jul 07 15:42:28.483168 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.483141 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-certificates\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:28.483502 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.483182 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:28.483502 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.483232 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfcr9\" (UniqueName: \"kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-kube-api-access-sfcr9\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:28.483502 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:28.483237 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:42:28.483502 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.483281 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-ca-trust-extracted\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:28.483502 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:28.483298 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73fd3204-d75e-41e7-ae19-734ddf9d80ae-metrics-tls podName:73fd3204-d75e-41e7-ae19-734ddf9d80ae nodeName:}" failed. No retries permitted until 2026-07-07 15:42:28.983278905 +0000 UTC m=+36.078559990 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/73fd3204-d75e-41e7-ae19-734ddf9d80ae-metrics-tls") pod "dns-default-m2fcv" (UID: "73fd3204-d75e-41e7-ae19-734ddf9d80ae") : secret "dns-default-metrics-tls" not found Jul 07 15:42:28.483502 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:28.483364 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 15:42:28.483502 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:28.483376 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-659cfff448-ggk7s: secret "image-registry-tls" not found Jul 07 15:42:28.483502 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:28.483441 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls podName:ffe49307-5a6f-4ba2-9fa7-f28af223ab94 nodeName:}" failed. No retries permitted until 2026-07-07 15:42:28.983425815 +0000 UTC m=+36.078706888 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls") pod "image-registry-659cfff448-ggk7s" (UID: "ffe49307-5a6f-4ba2-9fa7-f28af223ab94") : secret "image-registry-tls" not found Jul 07 15:42:28.484819 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.484789 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-trusted-ca\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:28.487426 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.487401 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-installation-pull-secrets\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:28.487524 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.487443 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19092adf-0ed9-4061-9a0a-bbc7dc75bc40-cert\") pod \"ingress-canary-xb8r5\" (UID: \"19092adf-0ed9-4061-9a0a-bbc7dc75bc40\") " pod="openshift-ingress-canary/ingress-canary-xb8r5" Jul 07 15:42:28.487582 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.487497 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-image-registry-private-configuration\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:28.500386 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.500190 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhwsr\" (UniqueName: \"kubernetes.io/projected/73fd3204-d75e-41e7-ae19-734ddf9d80ae-kube-api-access-lhwsr\") pod \"dns-default-m2fcv\" (UID: \"73fd3204-d75e-41e7-ae19-734ddf9d80ae\") " pod="openshift-dns/dns-default-m2fcv" Jul 07 15:42:28.500496 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.500456 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxvd9\" (UniqueName: \"kubernetes.io/projected/19092adf-0ed9-4061-9a0a-bbc7dc75bc40-kube-api-access-xxvd9\") pod \"ingress-canary-xb8r5\" (UID: \"19092adf-0ed9-4061-9a0a-bbc7dc75bc40\") " pod="openshift-ingress-canary/ingress-canary-xb8r5" Jul 07 15:42:28.500693 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.500671 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfcr9\" (UniqueName: \"kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-kube-api-access-sfcr9\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:28.501518 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.501494 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-bound-sa-token\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:28.543541 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.543512 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xb8r5" Jul 07 15:42:28.690085 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.689955 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xb8r5"] Jul 07 15:42:28.693319 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:42:28.693292 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19092adf_0ed9_4061_9a0a_bbc7dc75bc40.slice/crio-8e44279c750095ce953d03784b8ee8c76bc9d9f3f84c61e7385563cbe51be5c9 WatchSource:0}: Error finding container 8e44279c750095ce953d03784b8ee8c76bc9d9f3f84c61e7385563cbe51be5c9: Status 404 returned error can't find the container with id 8e44279c750095ce953d03784b8ee8c76bc9d9f3f84c61e7385563cbe51be5c9 Jul 07 15:42:28.987399 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.987314 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73fd3204-d75e-41e7-ae19-734ddf9d80ae-metrics-tls\") pod \"dns-default-m2fcv\" (UID: \"73fd3204-d75e-41e7-ae19-734ddf9d80ae\") " pod="openshift-dns/dns-default-m2fcv" Jul 07 15:42:28.987399 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:28.987371 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:28.987673 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:28.987476 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:42:28.987673 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:28.987526 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 15:42:28.987673 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:28.987539 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-659cfff448-ggk7s: secret "image-registry-tls" not found Jul 07 15:42:28.987673 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:28.987594 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls podName:ffe49307-5a6f-4ba2-9fa7-f28af223ab94 nodeName:}" failed. No retries permitted until 2026-07-07 15:42:29.987576743 +0000 UTC m=+37.082857815 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls") pod "image-registry-659cfff448-ggk7s" (UID: "ffe49307-5a6f-4ba2-9fa7-f28af223ab94") : secret "image-registry-tls" not found Jul 07 15:42:28.987673 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:28.987612 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73fd3204-d75e-41e7-ae19-734ddf9d80ae-metrics-tls podName:73fd3204-d75e-41e7-ae19-734ddf9d80ae nodeName:}" failed. No retries permitted until 2026-07-07 15:42:29.987603963 +0000 UTC m=+37.082885020 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/73fd3204-d75e-41e7-ae19-734ddf9d80ae-metrics-tls") pod "dns-default-m2fcv" (UID: "73fd3204-d75e-41e7-ae19-734ddf9d80ae") : secret "dns-default-metrics-tls" not found Jul 07 15:42:29.617375 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:29.617329 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xb8r5" event={"ID":"19092adf-0ed9-4061-9a0a-bbc7dc75bc40","Type":"ContainerStarted","Data":"8e44279c750095ce953d03784b8ee8c76bc9d9f3f84c61e7385563cbe51be5c9"} Jul 07 15:42:29.997343 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:29.997307 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73fd3204-d75e-41e7-ae19-734ddf9d80ae-metrics-tls\") pod \"dns-default-m2fcv\" (UID: \"73fd3204-d75e-41e7-ae19-734ddf9d80ae\") " pod="openshift-dns/dns-default-m2fcv" Jul 07 15:42:29.997520 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:29.997374 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:29.997520 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:29.997491 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:42:29.997520 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:29.997504 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 15:42:29.997520 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:29.997521 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-659cfff448-ggk7s: secret "image-registry-tls" not found Jul 07 15:42:29.997717 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:29.997568 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73fd3204-d75e-41e7-ae19-734ddf9d80ae-metrics-tls podName:73fd3204-d75e-41e7-ae19-734ddf9d80ae nodeName:}" failed. No retries permitted until 2026-07-07 15:42:31.997551126 +0000 UTC m=+39.092832187 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/73fd3204-d75e-41e7-ae19-734ddf9d80ae-metrics-tls") pod "dns-default-m2fcv" (UID: "73fd3204-d75e-41e7-ae19-734ddf9d80ae") : secret "dns-default-metrics-tls" not found Jul 07 15:42:29.997717 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:29.997583 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls podName:ffe49307-5a6f-4ba2-9fa7-f28af223ab94 nodeName:}" failed. No retries permitted until 2026-07-07 15:42:31.997577231 +0000 UTC m=+39.092858287 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls") pod "image-registry-659cfff448-ggk7s" (UID: "ffe49307-5a6f-4ba2-9fa7-f28af223ab94") : secret "image-registry-tls" not found Jul 07 15:42:31.626043 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:31.625761 2575 generic.go:356] "Generic (PLEG): container finished" podID="dd01334a-48d5-49a8-b01e-9f2956b5bf40" containerID="a0569dd5deb42ff839d5abd12ae3c36d72c400a00cc9b902a2f66106906e35f3" exitCode=0 Jul 07 15:42:31.626043 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:31.625852 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g42hc" event={"ID":"dd01334a-48d5-49a8-b01e-9f2956b5bf40","Type":"ContainerDied","Data":"a0569dd5deb42ff839d5abd12ae3c36d72c400a00cc9b902a2f66106906e35f3"} Jul 07 15:42:32.014280 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:32.014199 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73fd3204-d75e-41e7-ae19-734ddf9d80ae-metrics-tls\") pod \"dns-default-m2fcv\" (UID: \"73fd3204-d75e-41e7-ae19-734ddf9d80ae\") " pod="openshift-dns/dns-default-m2fcv" Jul 07 15:42:32.014280 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:32.014247 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:32.014482 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:32.014344 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 15:42:32.014482 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:32.014356 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-659cfff448-ggk7s: secret "image-registry-tls" not found Jul 07 15:42:32.014482 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:32.014401 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls podName:ffe49307-5a6f-4ba2-9fa7-f28af223ab94 nodeName:}" failed. No retries permitted until 2026-07-07 15:42:36.014388727 +0000 UTC m=+43.109669784 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls") pod "image-registry-659cfff448-ggk7s" (UID: "ffe49307-5a6f-4ba2-9fa7-f28af223ab94") : secret "image-registry-tls" not found Jul 07 15:42:32.014482 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:32.014344 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:42:32.014482 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:32.014463 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73fd3204-d75e-41e7-ae19-734ddf9d80ae-metrics-tls podName:73fd3204-d75e-41e7-ae19-734ddf9d80ae nodeName:}" failed. No retries permitted until 2026-07-07 15:42:36.014449823 +0000 UTC m=+43.109730884 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/73fd3204-d75e-41e7-ae19-734ddf9d80ae-metrics-tls") pod "dns-default-m2fcv" (UID: "73fd3204-d75e-41e7-ae19-734ddf9d80ae") : secret "dns-default-metrics-tls" not found Jul 07 15:42:32.633015 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:32.632978 2575 generic.go:356] "Generic (PLEG): container finished" podID="dd01334a-48d5-49a8-b01e-9f2956b5bf40" containerID="5948425d4ccad1056dfaa4b00fbc3f30a797668e336a54f47b97a728327dff86" exitCode=0 Jul 07 15:42:32.633446 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:32.633030 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g42hc" event={"ID":"dd01334a-48d5-49a8-b01e-9f2956b5bf40","Type":"ContainerDied","Data":"5948425d4ccad1056dfaa4b00fbc3f30a797668e336a54f47b97a728327dff86"} Jul 07 15:42:33.638022 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:33.637982 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xb8r5" event={"ID":"19092adf-0ed9-4061-9a0a-bbc7dc75bc40","Type":"ContainerStarted","Data":"22c66c28e7c6f02c876a4a70c45525a4ac33c39a377be4517a670c6e74a1a82c"} Jul 07 15:42:33.641266 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:33.641240 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g42hc" event={"ID":"dd01334a-48d5-49a8-b01e-9f2956b5bf40","Type":"ContainerStarted","Data":"3857ab3645bcb1f64f47644fa961149a6779e2bc19820147052d0052dca29db8"} Jul 07 15:42:33.655685 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:33.655637 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xb8r5" podStartSLOduration=1.341204931 podStartE2EDuration="5.655622829s" totalImagesPullingTime="4.314417898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:42:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:42:33.655362106 +0000 UTC m=+40.750643182" watchObservedRunningTime="2026-07-07 15:42:33.655622829 +0000 UTC m=+40.750903907" Jul 07 15:42:33.682540 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:33.682481 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-g42hc" podStartSLOduration=10.193681216 podStartE2EDuration="40.682449239s" totalImagesPullingTime="30.488768023s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:41:53 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:42:33.68057204 +0000 UTC m=+40.775853119" watchObservedRunningTime="2026-07-07 15:42:33.682449239 +0000 UTC m=+40.777730317" Jul 07 15:42:36.044729 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:36.044697 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73fd3204-d75e-41e7-ae19-734ddf9d80ae-metrics-tls\") pod \"dns-default-m2fcv\" (UID: \"73fd3204-d75e-41e7-ae19-734ddf9d80ae\") " pod="openshift-dns/dns-default-m2fcv" Jul 07 15:42:36.045116 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:36.044745 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:36.045116 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:36.044855 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:42:36.045116 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:36.044891 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 15:42:36.045116 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:36.044903 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-659cfff448-ggk7s: secret "image-registry-tls" not found Jul 07 15:42:36.045116 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:36.044938 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73fd3204-d75e-41e7-ae19-734ddf9d80ae-metrics-tls podName:73fd3204-d75e-41e7-ae19-734ddf9d80ae nodeName:}" failed. No retries permitted until 2026-07-07 15:42:44.044919226 +0000 UTC m=+51.140200296 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/73fd3204-d75e-41e7-ae19-734ddf9d80ae-metrics-tls") pod "dns-default-m2fcv" (UID: "73fd3204-d75e-41e7-ae19-734ddf9d80ae") : secret "dns-default-metrics-tls" not found Jul 07 15:42:36.045116 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:36.044952 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls podName:ffe49307-5a6f-4ba2-9fa7-f28af223ab94 nodeName:}" failed. No retries permitted until 2026-07-07 15:42:44.044946472 +0000 UTC m=+51.140227529 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls") pod "image-registry-659cfff448-ggk7s" (UID: "ffe49307-5a6f-4ba2-9fa7-f28af223ab94") : secret "image-registry-tls" not found Jul 07 15:42:44.101451 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:44.101407 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73fd3204-d75e-41e7-ae19-734ddf9d80ae-metrics-tls\") pod \"dns-default-m2fcv\" (UID: \"73fd3204-d75e-41e7-ae19-734ddf9d80ae\") " pod="openshift-dns/dns-default-m2fcv" Jul 07 15:42:44.101451 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:44.101460 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:42:44.102028 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:44.101552 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 15:42:44.102028 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:44.101552 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:42:44.102028 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:44.101624 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73fd3204-d75e-41e7-ae19-734ddf9d80ae-metrics-tls podName:73fd3204-d75e-41e7-ae19-734ddf9d80ae nodeName:}" failed. No retries permitted until 2026-07-07 15:43:00.101606738 +0000 UTC m=+67.196887795 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/73fd3204-d75e-41e7-ae19-734ddf9d80ae-metrics-tls") pod "dns-default-m2fcv" (UID: "73fd3204-d75e-41e7-ae19-734ddf9d80ae") : secret "dns-default-metrics-tls" not found Jul 07 15:42:44.102028 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:44.101563 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-659cfff448-ggk7s: secret "image-registry-tls" not found Jul 07 15:42:44.102028 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:44.101703 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls podName:ffe49307-5a6f-4ba2-9fa7-f28af223ab94 nodeName:}" failed. No retries permitted until 2026-07-07 15:43:00.101691521 +0000 UTC m=+67.196972582 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls") pod "image-registry-659cfff448-ggk7s" (UID: "ffe49307-5a6f-4ba2-9fa7-f28af223ab94") : secret "image-registry-tls" not found Jul 07 15:42:55.613546 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:55.613514 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4d86s" Jul 07 15:42:56.919749 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:56.919711 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-dvgc9"] Jul 07 15:42:56.922731 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:56.922713 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-dvgc9" Jul 07 15:42:56.925181 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:56.925160 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-t4xwj\"" Jul 07 15:42:56.930431 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:56.930406 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-nfh8b"] Jul 07 15:42:56.932665 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:56.932644 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-dvgc9"] Jul 07 15:42:56.932774 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:56.932728 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" Jul 07 15:42:56.940299 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:56.940273 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-vqcj5\"" Jul 07 15:42:56.940541 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:56.940517 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 07 15:42:56.940629 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:56.940559 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:42:56.940897 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:56.940879 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 07 15:42:56.940969 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:56.940903 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 07 15:42:56.950907 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:56.950888 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 07 15:42:56.953000 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:56.952888 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-nfh8b"] Jul 07 15:42:56.994605 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:56.994576 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flf4q\" (UniqueName: \"kubernetes.io/projected/162f95df-a98f-49a6-a91e-689a8e22be46-kube-api-access-flf4q\") pod \"console-operator-68b757fb67-nfh8b\" (UID: \"162f95df-a98f-49a6-a91e-689a8e22be46\") " pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" Jul 07 15:42:56.994775 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:56.994629 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/162f95df-a98f-49a6-a91e-689a8e22be46-serving-cert\") pod \"console-operator-68b757fb67-nfh8b\" (UID: \"162f95df-a98f-49a6-a91e-689a8e22be46\") " pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" Jul 07 15:42:56.994775 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:56.994671 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/162f95df-a98f-49a6-a91e-689a8e22be46-config\") pod \"console-operator-68b757fb67-nfh8b\" (UID: \"162f95df-a98f-49a6-a91e-689a8e22be46\") " pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" Jul 07 15:42:56.994775 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:56.994699 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nt2c\" (UniqueName: \"kubernetes.io/projected/ff69ad20-e6f4-4328-802b-67ff06013cbe-kube-api-access-8nt2c\") pod \"network-check-source-989f8b8dc-dvgc9\" (UID: \"ff69ad20-e6f4-4328-802b-67ff06013cbe\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-dvgc9" Jul 07 15:42:56.994775 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:56.994732 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/162f95df-a98f-49a6-a91e-689a8e22be46-trusted-ca\") pod \"console-operator-68b757fb67-nfh8b\" (UID: \"162f95df-a98f-49a6-a91e-689a8e22be46\") " pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" Jul 07 15:42:57.095284 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:57.095247 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/162f95df-a98f-49a6-a91e-689a8e22be46-serving-cert\") pod \"console-operator-68b757fb67-nfh8b\" (UID: \"162f95df-a98f-49a6-a91e-689a8e22be46\") " pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" Jul 07 15:42:57.095490 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:57.095291 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/162f95df-a98f-49a6-a91e-689a8e22be46-config\") pod \"console-operator-68b757fb67-nfh8b\" (UID: \"162f95df-a98f-49a6-a91e-689a8e22be46\") " pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" Jul 07 15:42:57.095490 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:57.095324 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nt2c\" (UniqueName: \"kubernetes.io/projected/ff69ad20-e6f4-4328-802b-67ff06013cbe-kube-api-access-8nt2c\") pod \"network-check-source-989f8b8dc-dvgc9\" (UID: \"ff69ad20-e6f4-4328-802b-67ff06013cbe\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-dvgc9" Jul 07 15:42:57.095490 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:57.095377 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/162f95df-a98f-49a6-a91e-689a8e22be46-trusted-ca\") pod \"console-operator-68b757fb67-nfh8b\" (UID: \"162f95df-a98f-49a6-a91e-689a8e22be46\") " pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" Jul 07 15:42:57.095490 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:57.095414 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flf4q\" (UniqueName: \"kubernetes.io/projected/162f95df-a98f-49a6-a91e-689a8e22be46-kube-api-access-flf4q\") pod \"console-operator-68b757fb67-nfh8b\" (UID: \"162f95df-a98f-49a6-a91e-689a8e22be46\") " pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" Jul 07 15:42:57.096168 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:57.096148 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/162f95df-a98f-49a6-a91e-689a8e22be46-config\") pod \"console-operator-68b757fb67-nfh8b\" (UID: \"162f95df-a98f-49a6-a91e-689a8e22be46\") " pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" Jul 07 15:42:57.096663 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:57.096641 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/162f95df-a98f-49a6-a91e-689a8e22be46-trusted-ca\") pod \"console-operator-68b757fb67-nfh8b\" (UID: \"162f95df-a98f-49a6-a91e-689a8e22be46\") " pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" Jul 07 15:42:57.099207 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:57.099184 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/162f95df-a98f-49a6-a91e-689a8e22be46-serving-cert\") pod \"console-operator-68b757fb67-nfh8b\" (UID: \"162f95df-a98f-49a6-a91e-689a8e22be46\") " pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" Jul 07 15:42:57.105338 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:57.105318 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 15:42:57.105559 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:57.105538 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flf4q\" (UniqueName: \"kubernetes.io/projected/162f95df-a98f-49a6-a91e-689a8e22be46-kube-api-access-flf4q\") pod \"console-operator-68b757fb67-nfh8b\" (UID: \"162f95df-a98f-49a6-a91e-689a8e22be46\") " pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" Jul 07 15:42:57.116452 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:57.116432 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 15:42:57.127006 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:57.126978 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nt2c\" (UniqueName: \"kubernetes.io/projected/ff69ad20-e6f4-4328-802b-67ff06013cbe-kube-api-access-8nt2c\") pod \"network-check-source-989f8b8dc-dvgc9\" (UID: \"ff69ad20-e6f4-4328-802b-67ff06013cbe\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-dvgc9" Jul 07 15:42:57.231315 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:57.231228 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-dvgc9" Jul 07 15:42:57.241535 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:57.241505 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" Jul 07 15:42:57.384302 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:57.384275 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-dvgc9"] Jul 07 15:42:57.386731 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:42:57.386705 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff69ad20_e6f4_4328_802b_67ff06013cbe.slice/crio-d96e054b0d63844b0d52ec5cfde3fb5b0d82932cb62cfc26d3fdf9b99d9609cb WatchSource:0}: Error finding container d96e054b0d63844b0d52ec5cfde3fb5b0d82932cb62cfc26d3fdf9b99d9609cb: Status 404 returned error can't find the container with id d96e054b0d63844b0d52ec5cfde3fb5b0d82932cb62cfc26d3fdf9b99d9609cb Jul 07 15:42:57.406190 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:57.406168 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-nfh8b"] Jul 07 15:42:57.408976 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:42:57.408955 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod162f95df_a98f_49a6_a91e_689a8e22be46.slice/crio-d6b7a9b1f975cc50693fcd4f1beb11c1fae711ab56c6ad2f5d656fd23dd3e780 WatchSource:0}: Error finding container d6b7a9b1f975cc50693fcd4f1beb11c1fae711ab56c6ad2f5d656fd23dd3e780: Status 404 returned error can't find the container with id d6b7a9b1f975cc50693fcd4f1beb11c1fae711ab56c6ad2f5d656fd23dd3e780 Jul 07 15:42:57.696965 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:57.696932 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" event={"ID":"162f95df-a98f-49a6-a91e-689a8e22be46","Type":"ContainerStarted","Data":"d6b7a9b1f975cc50693fcd4f1beb11c1fae711ab56c6ad2f5d656fd23dd3e780"} Jul 07 15:42:57.698014 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:57.697991 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-dvgc9" event={"ID":"ff69ad20-e6f4-4328-802b-67ff06013cbe","Type":"ContainerStarted","Data":"d96e054b0d63844b0d52ec5cfde3fb5b0d82932cb62cfc26d3fdf9b99d9609cb"} Jul 07 15:42:59.113726 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:59.113684 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs\") pod \"network-metrics-daemon-6p7tj\" (UID: \"39adfd20-ec8f-4114-ae3e-f6abc9c340f3\") " pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:42:59.118946 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:59.118916 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 15:42:59.124113 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:59.124070 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 07 15:42:59.124269 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:42:59.124160 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs podName:39adfd20-ec8f-4114-ae3e-f6abc9c340f3 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:03.124137404 +0000 UTC m=+130.219418461 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs") pod "network-metrics-daemon-6p7tj" (UID: "39adfd20-ec8f-4114-ae3e-f6abc9c340f3") : secret "metrics-daemon-secret" not found Jul 07 15:42:59.215011 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:59.214967 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e207ff37-6b1c-4da4-8327-54488837f74d-original-pull-secret\") pod \"global-pull-secret-syncer-nkjmf\" (UID: \"e207ff37-6b1c-4da4-8327-54488837f74d\") " pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:42:59.215216 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:59.215076 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwh5j\" (UniqueName: \"kubernetes.io/projected/1b09eb17-9cb5-42f8-9633-61378ad652d3-kube-api-access-qwh5j\") pod \"network-check-target-fwrrt\" (UID: \"1b09eb17-9cb5-42f8-9633-61378ad652d3\") " pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:42:59.218344 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:59.218314 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwh5j\" (UniqueName: \"kubernetes.io/projected/1b09eb17-9cb5-42f8-9633-61378ad652d3-kube-api-access-qwh5j\") pod \"network-check-target-fwrrt\" (UID: \"1b09eb17-9cb5-42f8-9633-61378ad652d3\") " pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:42:59.218656 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:59.218626 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 15:42:59.228518 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:59.228489 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e207ff37-6b1c-4da4-8327-54488837f74d-original-pull-secret\") pod \"global-pull-secret-syncer-nkjmf\" (UID: \"e207ff37-6b1c-4da4-8327-54488837f74d\") " pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:42:59.352846 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:59.352813 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-54b57\"" Jul 07 15:42:59.353890 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:59.353844 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkjmf" Jul 07 15:42:59.360439 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:59.360419 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:42:59.778714 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:59.778683 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fwrrt"] Jul 07 15:42:59.781733 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:42:59.781710 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b09eb17_9cb5_42f8_9633_61378ad652d3.slice/crio-004d9564ac8b22bb7207c8d9f3b26fdc649ffdf5d1d118fec42a98176f42b1b6 WatchSource:0}: Error finding container 004d9564ac8b22bb7207c8d9f3b26fdc649ffdf5d1d118fec42a98176f42b1b6: Status 404 returned error can't find the container with id 004d9564ac8b22bb7207c8d9f3b26fdc649ffdf5d1d118fec42a98176f42b1b6 Jul 07 15:42:59.794373 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:42:59.794305 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nkjmf"] Jul 07 15:42:59.797577 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:42:59.797552 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode207ff37_6b1c_4da4_8327_54488837f74d.slice/crio-57a9b277f3bfc6cd991b9ed6d907bd164c675e83db5a50984b0d13da14003178 WatchSource:0}: Error finding container 57a9b277f3bfc6cd991b9ed6d907bd164c675e83db5a50984b0d13da14003178: Status 404 returned error can't find the container with id 57a9b277f3bfc6cd991b9ed6d907bd164c675e83db5a50984b0d13da14003178 Jul 07 15:43:00.123701 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:00.123667 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73fd3204-d75e-41e7-ae19-734ddf9d80ae-metrics-tls\") pod \"dns-default-m2fcv\" (UID: \"73fd3204-d75e-41e7-ae19-734ddf9d80ae\") " pod="openshift-dns/dns-default-m2fcv" Jul 07 15:43:00.124150 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:00.123714 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:43:00.124150 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:00.123835 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:43:00.124150 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:00.123856 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 15:43:00.124150 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:00.123888 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-659cfff448-ggk7s: secret "image-registry-tls" not found Jul 07 15:43:00.124150 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:00.123931 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73fd3204-d75e-41e7-ae19-734ddf9d80ae-metrics-tls podName:73fd3204-d75e-41e7-ae19-734ddf9d80ae nodeName:}" failed. No retries permitted until 2026-07-07 15:43:32.123907511 +0000 UTC m=+99.219188587 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/73fd3204-d75e-41e7-ae19-734ddf9d80ae-metrics-tls") pod "dns-default-m2fcv" (UID: "73fd3204-d75e-41e7-ae19-734ddf9d80ae") : secret "dns-default-metrics-tls" not found Jul 07 15:43:00.124150 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:00.123953 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls podName:ffe49307-5a6f-4ba2-9fa7-f28af223ab94 nodeName:}" failed. No retries permitted until 2026-07-07 15:43:32.123942712 +0000 UTC m=+99.219223772 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls") pod "image-registry-659cfff448-ggk7s" (UID: "ffe49307-5a6f-4ba2-9fa7-f28af223ab94") : secret "image-registry-tls" not found Jul 07 15:43:00.707590 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:00.707550 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fwrrt" event={"ID":"1b09eb17-9cb5-42f8-9633-61378ad652d3","Type":"ContainerStarted","Data":"004d9564ac8b22bb7207c8d9f3b26fdc649ffdf5d1d118fec42a98176f42b1b6"} Jul 07 15:43:00.708927 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:00.708896 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nkjmf" event={"ID":"e207ff37-6b1c-4da4-8327-54488837f74d","Type":"ContainerStarted","Data":"57a9b277f3bfc6cd991b9ed6d907bd164c675e83db5a50984b0d13da14003178"} Jul 07 15:43:00.710831 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:00.710807 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nfh8b_162f95df-a98f-49a6-a91e-689a8e22be46/console-operator/0.log" Jul 07 15:43:00.710966 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:00.710852 2575 generic.go:356] "Generic (PLEG): container finished" podID="162f95df-a98f-49a6-a91e-689a8e22be46" containerID="b0c9d3f2b360bfb3c22a95b4ba7e8d4cde309922cfd43625c79eba86e413c925" exitCode=255 Jul 07 15:43:00.710966 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:00.710913 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" event={"ID":"162f95df-a98f-49a6-a91e-689a8e22be46","Type":"ContainerDied","Data":"b0c9d3f2b360bfb3c22a95b4ba7e8d4cde309922cfd43625c79eba86e413c925"} Jul 07 15:43:00.711189 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:00.711169 2575 scope.go:117] "RemoveContainer" containerID="b0c9d3f2b360bfb3c22a95b4ba7e8d4cde309922cfd43625c79eba86e413c925" Jul 07 15:43:01.720618 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:01.720583 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nfh8b_162f95df-a98f-49a6-a91e-689a8e22be46/console-operator/1.log" Jul 07 15:43:01.721124 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:01.721064 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nfh8b_162f95df-a98f-49a6-a91e-689a8e22be46/console-operator/0.log" Jul 07 15:43:01.721124 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:01.721104 2575 generic.go:356] "Generic (PLEG): container finished" podID="162f95df-a98f-49a6-a91e-689a8e22be46" containerID="871ea39a7788e9b3e8846699d531b1d10738ebaf5517628cc3db4d7db3860540" exitCode=255 Jul 07 15:43:01.721299 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:01.721199 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" event={"ID":"162f95df-a98f-49a6-a91e-689a8e22be46","Type":"ContainerDied","Data":"871ea39a7788e9b3e8846699d531b1d10738ebaf5517628cc3db4d7db3860540"} Jul 07 15:43:01.721299 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:01.721270 2575 scope.go:117] "RemoveContainer" containerID="b0c9d3f2b360bfb3c22a95b4ba7e8d4cde309922cfd43625c79eba86e413c925" Jul 07 15:43:01.721596 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:01.721572 2575 scope.go:117] "RemoveContainer" containerID="871ea39a7788e9b3e8846699d531b1d10738ebaf5517628cc3db4d7db3860540" Jul 07 15:43:01.721842 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:01.721814 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-nfh8b_openshift-console-operator(162f95df-a98f-49a6-a91e-689a8e22be46)\"" pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" podUID="162f95df-a98f-49a6-a91e-689a8e22be46" Jul 07 15:43:01.723086 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:01.722897 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fwrrt" event={"ID":"1b09eb17-9cb5-42f8-9633-61378ad652d3","Type":"ContainerStarted","Data":"83db8a50818c128d656a3a7eb86b994453bda1bfc5cc37c83fc52c32eac08bdb"} Jul 07 15:43:01.723086 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:01.723048 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:43:01.724631 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:01.724609 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-dvgc9" event={"ID":"ff69ad20-e6f4-4328-802b-67ff06013cbe","Type":"ContainerStarted","Data":"3cd5787e2a6aae46440326946506e45980b209c839dd8213dac7ba85f71da5c5"} Jul 07 15:43:01.821365 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:01.821311 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-dvgc9" podStartSLOduration=2.239465128 podStartE2EDuration="5.821295714s" totalImagesPullingTime="3.581830586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:42:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:43:01.786885377 +0000 UTC m=+68.882166457" watchObservedRunningTime="2026-07-07 15:43:01.821295714 +0000 UTC m=+68.916576828" Jul 07 15:43:01.821903 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:01.821844 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fwrrt" podStartSLOduration=67.634505597 podStartE2EDuration="1m8.821834103s" totalImagesPullingTime="1.187328506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:43:01.821083224 +0000 UTC m=+68.916364308" watchObservedRunningTime="2026-07-07 15:43:01.821834103 +0000 UTC m=+68.917115180" Jul 07 15:43:02.729552 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:02.729520 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nfh8b_162f95df-a98f-49a6-a91e-689a8e22be46/console-operator/1.log" Jul 07 15:43:02.730038 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:02.729926 2575 scope.go:117] "RemoveContainer" containerID="871ea39a7788e9b3e8846699d531b1d10738ebaf5517628cc3db4d7db3860540" Jul 07 15:43:02.730128 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:02.730106 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-nfh8b_openshift-console-operator(162f95df-a98f-49a6-a91e-689a8e22be46)\"" pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" podUID="162f95df-a98f-49a6-a91e-689a8e22be46" Jul 07 15:43:03.266287 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.266254 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tv4kj"] Jul 07 15:43:03.288256 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.288214 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:03.288606 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.288535 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tv4kj"] Jul 07 15:43:03.294533 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.294507 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 15:43:03.294651 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.294554 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 15:43:03.294997 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.294810 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 15:43:03.295670 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.295643 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 15:43:03.296555 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.296534 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-68vpx\"" Jul 07 15:43:03.352302 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.352268 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tv4kj\" (UID: \"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953\") " pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:03.352302 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.352303 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tv4kj\" (UID: \"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953\") " pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:03.352541 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.352348 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-data-volume\") pod \"insights-runtime-extractor-tv4kj\" (UID: \"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953\") " pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:03.352541 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.352424 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slxrs\" (UniqueName: \"kubernetes.io/projected/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-kube-api-access-slxrs\") pod \"insights-runtime-extractor-tv4kj\" (UID: \"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953\") " pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:03.352541 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.352460 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-crio-socket\") pod \"insights-runtime-extractor-tv4kj\" (UID: \"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953\") " pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:03.362369 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.362337 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-dx8vc"] Jul 07 15:43:03.380302 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.380275 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dx8vc" Jul 07 15:43:03.382724 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.382702 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-dx8vc"] Jul 07 15:43:03.383294 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.383274 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-9b5w8\"" Jul 07 15:43:03.383986 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.383963 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 07 15:43:03.384080 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.384001 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 07 15:43:03.452794 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.452764 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tv4kj\" (UID: \"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953\") " pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:03.453033 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.452802 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tv4kj\" (UID: \"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953\") " pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:03.453033 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.452856 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-data-volume\") pod \"insights-runtime-extractor-tv4kj\" (UID: \"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953\") " pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:03.453033 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:03.452923 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 15:43:03.453033 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.452941 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slxrs\" (UniqueName: \"kubernetes.io/projected/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-kube-api-access-slxrs\") pod \"insights-runtime-extractor-tv4kj\" (UID: \"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953\") " pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:03.453033 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.452974 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk67q\" (UniqueName: \"kubernetes.io/projected/da62451e-981a-4473-85a5-ec61606dc856-kube-api-access-qk67q\") pod \"migrator-d57d58cc9-dx8vc\" (UID: \"da62451e-981a-4473-85a5-ec61606dc856\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dx8vc" Jul 07 15:43:03.453033 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:03.453022 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-kube-rbac-proxy-cm podName:2b5661bd-f1c4-4f6e-89a8-6e84a49fb953 nodeName:}" failed. No retries permitted until 2026-07-07 15:43:03.952999599 +0000 UTC m=+71.048280677 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tv4kj" (UID: "2b5661bd-f1c4-4f6e-89a8-6e84a49fb953") : configmap "kube-rbac-proxy" not found Jul 07 15:43:03.453033 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:03.453027 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 15:43:03.453355 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:03.453232 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-insights-runtime-extractor-tls podName:2b5661bd-f1c4-4f6e-89a8-6e84a49fb953 nodeName:}" failed. No retries permitted until 2026-07-07 15:43:03.953215769 +0000 UTC m=+71.048496841 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tv4kj" (UID: "2b5661bd-f1c4-4f6e-89a8-6e84a49fb953") : secret "insights-runtime-extractor-tls" not found Jul 07 15:43:03.453355 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.453252 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-crio-socket\") pod \"insights-runtime-extractor-tv4kj\" (UID: \"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953\") " pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:03.453355 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.453330 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-data-volume\") pod \"insights-runtime-extractor-tv4kj\" (UID: \"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953\") " pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:03.453355 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.453347 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-crio-socket\") pod \"insights-runtime-extractor-tv4kj\" (UID: \"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953\") " pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:03.476385 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.476351 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slxrs\" (UniqueName: \"kubernetes.io/projected/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-kube-api-access-slxrs\") pod \"insights-runtime-extractor-tv4kj\" (UID: \"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953\") " pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:03.554243 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.554163 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk67q\" (UniqueName: \"kubernetes.io/projected/da62451e-981a-4473-85a5-ec61606dc856-kube-api-access-qk67q\") pod \"migrator-d57d58cc9-dx8vc\" (UID: \"da62451e-981a-4473-85a5-ec61606dc856\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dx8vc" Jul 07 15:43:03.580936 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.580902 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk67q\" (UniqueName: \"kubernetes.io/projected/da62451e-981a-4473-85a5-ec61606dc856-kube-api-access-qk67q\") pod \"migrator-d57d58cc9-dx8vc\" (UID: \"da62451e-981a-4473-85a5-ec61606dc856\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dx8vc" Jul 07 15:43:03.691156 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.691118 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dx8vc" Jul 07 15:43:03.957291 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.957251 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tv4kj\" (UID: \"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953\") " pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:03.957291 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:03.957296 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tv4kj\" (UID: \"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953\") " pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:03.957825 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:03.957385 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 15:43:03.957825 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:03.957402 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 15:43:03.957825 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:03.957467 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-kube-rbac-proxy-cm podName:2b5661bd-f1c4-4f6e-89a8-6e84a49fb953 nodeName:}" failed. No retries permitted until 2026-07-07 15:43:04.957444005 +0000 UTC m=+72.052725079 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tv4kj" (UID: "2b5661bd-f1c4-4f6e-89a8-6e84a49fb953") : configmap "kube-rbac-proxy" not found Jul 07 15:43:03.957825 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:03.957489 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-insights-runtime-extractor-tls podName:2b5661bd-f1c4-4f6e-89a8-6e84a49fb953 nodeName:}" failed. No retries permitted until 2026-07-07 15:43:04.95747994 +0000 UTC m=+72.052761003 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tv4kj" (UID: "2b5661bd-f1c4-4f6e-89a8-6e84a49fb953") : secret "insights-runtime-extractor-tls" not found Jul 07 15:43:04.603337 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:04.603234 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-dx8vc"] Jul 07 15:43:04.605723 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:43:04.605683 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda62451e_981a_4473_85a5_ec61606dc856.slice/crio-280cc6df3be4f24f40a2f0be7a97c0df193cbf20c94c8da5722a3f89d7d8afb9 WatchSource:0}: Error finding container 280cc6df3be4f24f40a2f0be7a97c0df193cbf20c94c8da5722a3f89d7d8afb9: Status 404 returned error can't find the container with id 280cc6df3be4f24f40a2f0be7a97c0df193cbf20c94c8da5722a3f89d7d8afb9 Jul 07 15:43:04.735909 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:04.735809 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dx8vc" event={"ID":"da62451e-981a-4473-85a5-ec61606dc856","Type":"ContainerStarted","Data":"280cc6df3be4f24f40a2f0be7a97c0df193cbf20c94c8da5722a3f89d7d8afb9"} Jul 07 15:43:04.737267 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:04.737239 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nkjmf" event={"ID":"e207ff37-6b1c-4da4-8327-54488837f74d","Type":"ContainerStarted","Data":"5069690b916c9aaf84a8215914810d11b9dbad393843afa8a8d4058716bd5c52"} Jul 07 15:43:04.762201 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:04.762146 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nkjmf" podStartSLOduration=67.051170349 podStartE2EDuration="1m11.762129305s" totalImagesPullingTime="4.710958956s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:43:04.760516168 +0000 UTC m=+71.855797251" watchObservedRunningTime="2026-07-07 15:43:04.762129305 +0000 UTC m=+71.857410384" Jul 07 15:43:04.965785 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:04.965744 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tv4kj\" (UID: \"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953\") " pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:04.965785 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:04.965784 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tv4kj\" (UID: \"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953\") " pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:04.966241 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:04.965896 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 15:43:04.966241 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:04.965930 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 15:43:04.966241 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:04.965968 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-kube-rbac-proxy-cm podName:2b5661bd-f1c4-4f6e-89a8-6e84a49fb953 nodeName:}" failed. No retries permitted until 2026-07-07 15:43:06.965949461 +0000 UTC m=+74.061230538 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tv4kj" (UID: "2b5661bd-f1c4-4f6e-89a8-6e84a49fb953") : configmap "kube-rbac-proxy" not found Jul 07 15:43:04.966241 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:04.965984 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-insights-runtime-extractor-tls podName:2b5661bd-f1c4-4f6e-89a8-6e84a49fb953 nodeName:}" failed. No retries permitted until 2026-07-07 15:43:06.965976755 +0000 UTC m=+74.061257812 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tv4kj" (UID: "2b5661bd-f1c4-4f6e-89a8-6e84a49fb953") : secret "insights-runtime-extractor-tls" not found Jul 07 15:43:05.063748 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:05.063671 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mpvp5_56652ce6-f2fe-44ee-a263-72343b20fafe/dns-node-resolver/0.log" Jul 07 15:43:06.266266 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:06.266237 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9ng9d_0efee2cb-622f-4f59-a8a4-1eecfc38ead4/node-ca/0.log" Jul 07 15:43:06.744762 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:06.744722 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dx8vc" event={"ID":"da62451e-981a-4473-85a5-ec61606dc856","Type":"ContainerStarted","Data":"2614f2c98dc3a7747a8db32c31f06d8946deb552d5fdbdde22efe2f2ce589ecd"} Jul 07 15:43:06.744948 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:06.744777 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dx8vc" event={"ID":"da62451e-981a-4473-85a5-ec61606dc856","Type":"ContainerStarted","Data":"79635de0e5a7872d2712929a1704f1491fd7d621898d984c62e7a40e2531026e"} Jul 07 15:43:06.767552 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:06.767505 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dx8vc" podStartSLOduration=2.01102071 podStartE2EDuration="3.767488977s" totalImagesPullingTime="1.756468267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:43:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:43:06.766656944 +0000 UTC m=+73.861938026" watchObservedRunningTime="2026-07-07 15:43:06.767488977 +0000 UTC m=+73.862770060" Jul 07 15:43:06.981197 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:06.981158 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tv4kj\" (UID: \"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953\") " pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:06.981197 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:06.981198 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tv4kj\" (UID: \"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953\") " pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:06.981401 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:06.981285 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 15:43:06.981401 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:06.981342 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 15:43:06.981401 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:06.981354 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-kube-rbac-proxy-cm podName:2b5661bd-f1c4-4f6e-89a8-6e84a49fb953 nodeName:}" failed. No retries permitted until 2026-07-07 15:43:10.981338859 +0000 UTC m=+78.076619919 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tv4kj" (UID: "2b5661bd-f1c4-4f6e-89a8-6e84a49fb953") : configmap "kube-rbac-proxy" not found Jul 07 15:43:06.981401 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:06.981385 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-insights-runtime-extractor-tls podName:2b5661bd-f1c4-4f6e-89a8-6e84a49fb953 nodeName:}" failed. No retries permitted until 2026-07-07 15:43:10.981373819 +0000 UTC m=+78.076654876 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tv4kj" (UID: "2b5661bd-f1c4-4f6e-89a8-6e84a49fb953") : secret "insights-runtime-extractor-tls" not found Jul 07 15:43:07.242655 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:07.242607 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" Jul 07 15:43:07.242822 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:07.242678 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" Jul 07 15:43:07.243027 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:07.243015 2575 scope.go:117] "RemoveContainer" containerID="871ea39a7788e9b3e8846699d531b1d10738ebaf5517628cc3db4d7db3860540" Jul 07 15:43:07.243168 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:07.243150 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-nfh8b_openshift-console-operator(162f95df-a98f-49a6-a91e-689a8e22be46)\"" pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" podUID="162f95df-a98f-49a6-a91e-689a8e22be46" Jul 07 15:43:07.270851 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:07.270823 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xb8r5_19092adf-0ed9-4061-9a0a-bbc7dc75bc40/serve-healthcheck-canary/0.log" Jul 07 15:43:11.015480 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:11.015432 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tv4kj\" (UID: \"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953\") " pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:11.015480 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:11.015482 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tv4kj\" (UID: \"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953\") " pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:11.015959 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:11.015549 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 15:43:11.015959 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:11.015604 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 15:43:11.015959 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:11.015617 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-kube-rbac-proxy-cm podName:2b5661bd-f1c4-4f6e-89a8-6e84a49fb953 nodeName:}" failed. No retries permitted until 2026-07-07 15:43:19.015600753 +0000 UTC m=+86.110881811 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tv4kj" (UID: "2b5661bd-f1c4-4f6e-89a8-6e84a49fb953") : configmap "kube-rbac-proxy" not found Jul 07 15:43:11.015959 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:11.015644 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-insights-runtime-extractor-tls podName:2b5661bd-f1c4-4f6e-89a8-6e84a49fb953 nodeName:}" failed. No retries permitted until 2026-07-07 15:43:19.015633655 +0000 UTC m=+86.110914711 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tv4kj" (UID: "2b5661bd-f1c4-4f6e-89a8-6e84a49fb953") : secret "insights-runtime-extractor-tls" not found Jul 07 15:43:19.078501 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:19.078363 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tv4kj\" (UID: \"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953\") " pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:19.078501 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:19.078414 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tv4kj\" (UID: \"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953\") " pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:19.078953 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:19.078890 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tv4kj\" (UID: \"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953\") " pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:19.080671 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:19.080653 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b5661bd-f1c4-4f6e-89a8-6e84a49fb953-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tv4kj\" (UID: \"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953\") " pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:19.199177 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:19.199140 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tv4kj" Jul 07 15:43:19.325421 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:19.325394 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tv4kj"] Jul 07 15:43:19.327966 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:43:19.327940 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b5661bd_f1c4_4f6e_89a8_6e84a49fb953.slice/crio-c71ad2a07136d1a45a994474e3779797b6385e64fdd33c3b547808cec86e6c95 WatchSource:0}: Error finding container c71ad2a07136d1a45a994474e3779797b6385e64fdd33c3b547808cec86e6c95: Status 404 returned error can't find the container with id c71ad2a07136d1a45a994474e3779797b6385e64fdd33c3b547808cec86e6c95 Jul 07 15:43:19.430364 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:19.430333 2575 scope.go:117] "RemoveContainer" containerID="871ea39a7788e9b3e8846699d531b1d10738ebaf5517628cc3db4d7db3860540" Jul 07 15:43:19.783946 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:19.783919 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nfh8b_162f95df-a98f-49a6-a91e-689a8e22be46/console-operator/1.log" Jul 07 15:43:19.784111 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:19.784040 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" event={"ID":"162f95df-a98f-49a6-a91e-689a8e22be46","Type":"ContainerStarted","Data":"1f74e36dfd98c082049ad420ab8e2e991d45e7be95aeb2b2fd2eff9130d2bba3"} Jul 07 15:43:19.784368 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:19.784340 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" Jul 07 15:43:19.785578 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:19.785548 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tv4kj" event={"ID":"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953","Type":"ContainerStarted","Data":"c71ad2a07136d1a45a994474e3779797b6385e64fdd33c3b547808cec86e6c95"} Jul 07 15:43:19.805910 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:19.805847 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" podStartSLOduration=21.553229212 podStartE2EDuration="23.805832808s" totalImagesPullingTime="2.252603596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:42:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:43:19.804932306 +0000 UTC m=+86.900213383" watchObservedRunningTime="2026-07-07 15:43:19.805832808 +0000 UTC m=+86.901114228" Jul 07 15:43:19.897784 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:19.897751 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-nfh8b" Jul 07 15:43:23.801056 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:23.801012 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tv4kj" event={"ID":"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953","Type":"ContainerStarted","Data":"4491dfab5ca43df84104bed26eca415ca896dd14defbf760195b3c5c2ea0b54f"} Jul 07 15:43:24.806851 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:24.806812 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tv4kj" event={"ID":"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953","Type":"ContainerStarted","Data":"7e04218d88801a2b7b5ea2c68f22ca6237859e724e3d75d62f66f73f9cc7bd19"} Jul 07 15:43:24.844392 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:24.844356 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68874744f8-xw5x5"] Jul 07 15:43:24.846352 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:24.846328 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68874744f8-xw5x5" Jul 07 15:43:24.849094 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:24.849067 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 07 15:43:24.850432 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:24.850317 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-vqlhz\"" Jul 07 15:43:24.850813 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:24.850791 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 07 15:43:24.850813 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:24.850814 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 07 15:43:24.851108 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:24.851092 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 07 15:43:24.871703 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:24.870038 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68874744f8-xw5x5"] Jul 07 15:43:24.925794 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:24.925765 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mglqg\" (UniqueName: \"kubernetes.io/projected/63e08479-7dd2-4611-ac78-a44090d9ec54-kube-api-access-mglqg\") pod \"managed-serviceaccount-addon-agent-68874744f8-xw5x5\" (UID: \"63e08479-7dd2-4611-ac78-a44090d9ec54\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68874744f8-xw5x5" Jul 07 15:43:24.925989 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:24.925804 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/63e08479-7dd2-4611-ac78-a44090d9ec54-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68874744f8-xw5x5\" (UID: \"63e08479-7dd2-4611-ac78-a44090d9ec54\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68874744f8-xw5x5" Jul 07 15:43:25.027329 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:25.027277 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/63e08479-7dd2-4611-ac78-a44090d9ec54-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68874744f8-xw5x5\" (UID: \"63e08479-7dd2-4611-ac78-a44090d9ec54\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68874744f8-xw5x5" Jul 07 15:43:25.027509 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:25.027422 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mglqg\" (UniqueName: \"kubernetes.io/projected/63e08479-7dd2-4611-ac78-a44090d9ec54-kube-api-access-mglqg\") pod \"managed-serviceaccount-addon-agent-68874744f8-xw5x5\" (UID: \"63e08479-7dd2-4611-ac78-a44090d9ec54\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68874744f8-xw5x5" Jul 07 15:43:25.030172 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:25.030141 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/63e08479-7dd2-4611-ac78-a44090d9ec54-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68874744f8-xw5x5\" (UID: \"63e08479-7dd2-4611-ac78-a44090d9ec54\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68874744f8-xw5x5" Jul 07 15:43:25.038208 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:25.038181 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mglqg\" (UniqueName: \"kubernetes.io/projected/63e08479-7dd2-4611-ac78-a44090d9ec54-kube-api-access-mglqg\") pod \"managed-serviceaccount-addon-agent-68874744f8-xw5x5\" (UID: \"63e08479-7dd2-4611-ac78-a44090d9ec54\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68874744f8-xw5x5" Jul 07 15:43:25.165583 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:25.165546 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68874744f8-xw5x5" Jul 07 15:43:25.311791 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:25.311755 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68874744f8-xw5x5"] Jul 07 15:43:25.314984 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:43:25.314942 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63e08479_7dd2_4611_ac78_a44090d9ec54.slice/crio-dda275eae1426de53831a937e32c9dd7461f3634f4fadb8dd82018fdca0ef0a2 WatchSource:0}: Error finding container dda275eae1426de53831a937e32c9dd7461f3634f4fadb8dd82018fdca0ef0a2: Status 404 returned error can't find the container with id dda275eae1426de53831a937e32c9dd7461f3634f4fadb8dd82018fdca0ef0a2 Jul 07 15:43:25.811646 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:25.811610 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68874744f8-xw5x5" event={"ID":"63e08479-7dd2-4611-ac78-a44090d9ec54","Type":"ContainerStarted","Data":"dda275eae1426de53831a937e32c9dd7461f3634f4fadb8dd82018fdca0ef0a2"} Jul 07 15:43:26.816123 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:26.816086 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tv4kj" event={"ID":"2b5661bd-f1c4-4f6e-89a8-6e84a49fb953","Type":"ContainerStarted","Data":"669937ba50b54ed8443fe6dbf7c041256d4a340afb8242b77f5d5b76ff521e73"} Jul 07 15:43:26.837621 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:26.837559 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tv4kj" podStartSLOduration=16.604704935 podStartE2EDuration="23.837536538s" totalImagesPullingTime="7.232831603s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:43:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:43:26.835931771 +0000 UTC m=+93.931212850" watchObservedRunningTime="2026-07-07 15:43:26.837536538 +0000 UTC m=+93.932817619" Jul 07 15:43:28.825439 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:28.825354 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68874744f8-xw5x5" event={"ID":"63e08479-7dd2-4611-ac78-a44090d9ec54","Type":"ContainerStarted","Data":"2f6174e8726397896c524bbbfb32b3dfb7f59bbfa31f85cffe6cc58e77ca1635"} Jul 07 15:43:28.844336 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:28.844283 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68874744f8-xw5x5" podStartSLOduration=1.595120615 podStartE2EDuration="4.844267608s" totalImagesPullingTime="3.249146993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:43:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:43:28.843711213 +0000 UTC m=+95.938992293" watchObservedRunningTime="2026-07-07 15:43:28.844267608 +0000 UTC m=+95.939548715" Jul 07 15:43:29.649845 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:29.649809 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-2cld9"] Jul 07 15:43:29.652413 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:29.652397 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-2cld9" Jul 07 15:43:29.656857 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:29.656828 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 15:43:29.657540 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:29.657509 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 15:43:29.657659 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:29.657589 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 07 15:43:29.657659 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:29.657625 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-r65fq\"" Jul 07 15:43:29.657659 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:29.657652 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 15:43:29.657966 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:29.657949 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 07 15:43:29.667622 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:29.667598 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-2cld9"] Jul 07 15:43:29.768858 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:29.768821 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fc8fb5d-775b-4cf5-929d-21354c456e1b-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-2cld9\" (UID: \"0fc8fb5d-775b-4cf5-929d-21354c456e1b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-2cld9" Jul 07 15:43:29.769027 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:29.768913 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fc8fb5d-775b-4cf5-929d-21354c456e1b-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-2cld9\" (UID: \"0fc8fb5d-775b-4cf5-929d-21354c456e1b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-2cld9" Jul 07 15:43:29.769060 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:29.769031 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0fc8fb5d-775b-4cf5-929d-21354c456e1b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-2cld9\" (UID: \"0fc8fb5d-775b-4cf5-929d-21354c456e1b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-2cld9" Jul 07 15:43:29.769113 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:29.769060 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqhpp\" (UniqueName: \"kubernetes.io/projected/0fc8fb5d-775b-4cf5-929d-21354c456e1b-kube-api-access-tqhpp\") pod \"prometheus-operator-567f7b8b7c-2cld9\" (UID: \"0fc8fb5d-775b-4cf5-929d-21354c456e1b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-2cld9" Jul 07 15:43:29.870150 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:29.870107 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0fc8fb5d-775b-4cf5-929d-21354c456e1b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-2cld9\" (UID: \"0fc8fb5d-775b-4cf5-929d-21354c456e1b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-2cld9" Jul 07 15:43:29.870617 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:29.870148 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqhpp\" (UniqueName: \"kubernetes.io/projected/0fc8fb5d-775b-4cf5-929d-21354c456e1b-kube-api-access-tqhpp\") pod \"prometheus-operator-567f7b8b7c-2cld9\" (UID: \"0fc8fb5d-775b-4cf5-929d-21354c456e1b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-2cld9" Jul 07 15:43:29.870617 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:29.870186 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fc8fb5d-775b-4cf5-929d-21354c456e1b-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-2cld9\" (UID: \"0fc8fb5d-775b-4cf5-929d-21354c456e1b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-2cld9" Jul 07 15:43:29.870617 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:29.870229 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fc8fb5d-775b-4cf5-929d-21354c456e1b-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-2cld9\" (UID: \"0fc8fb5d-775b-4cf5-929d-21354c456e1b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-2cld9" Jul 07 15:43:29.871018 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:29.870996 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fc8fb5d-775b-4cf5-929d-21354c456e1b-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-2cld9\" (UID: \"0fc8fb5d-775b-4cf5-929d-21354c456e1b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-2cld9" Jul 07 15:43:29.872605 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:29.872575 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0fc8fb5d-775b-4cf5-929d-21354c456e1b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-2cld9\" (UID: \"0fc8fb5d-775b-4cf5-929d-21354c456e1b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-2cld9" Jul 07 15:43:29.872699 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:29.872659 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fc8fb5d-775b-4cf5-929d-21354c456e1b-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-2cld9\" (UID: \"0fc8fb5d-775b-4cf5-929d-21354c456e1b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-2cld9" Jul 07 15:43:29.882771 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:29.882748 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqhpp\" (UniqueName: \"kubernetes.io/projected/0fc8fb5d-775b-4cf5-929d-21354c456e1b-kube-api-access-tqhpp\") pod \"prometheus-operator-567f7b8b7c-2cld9\" (UID: \"0fc8fb5d-775b-4cf5-929d-21354c456e1b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-2cld9" Jul 07 15:43:29.961944 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:29.961851 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-2cld9" Jul 07 15:43:30.100816 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:30.100786 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-2cld9"] Jul 07 15:43:30.103371 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:43:30.103346 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fc8fb5d_775b_4cf5_929d_21354c456e1b.slice/crio-7718fa690c4cbc98df8556c856df92a7e218182b4b17394c3d981280b8c39a9e WatchSource:0}: Error finding container 7718fa690c4cbc98df8556c856df92a7e218182b4b17394c3d981280b8c39a9e: Status 404 returned error can't find the container with id 7718fa690c4cbc98df8556c856df92a7e218182b4b17394c3d981280b8c39a9e Jul 07 15:43:30.834023 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:30.833976 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-2cld9" event={"ID":"0fc8fb5d-775b-4cf5-929d-21354c456e1b","Type":"ContainerStarted","Data":"7718fa690c4cbc98df8556c856df92a7e218182b4b17394c3d981280b8c39a9e"} Jul 07 15:43:31.838261 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:31.838231 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-2cld9" event={"ID":"0fc8fb5d-775b-4cf5-929d-21354c456e1b","Type":"ContainerStarted","Data":"1aed47f50290fea6de3a539197b34d8baf9014873d43aeaedc3d30df382efd8a"} Jul 07 15:43:32.189771 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:32.189729 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73fd3204-d75e-41e7-ae19-734ddf9d80ae-metrics-tls\") pod \"dns-default-m2fcv\" (UID: \"73fd3204-d75e-41e7-ae19-734ddf9d80ae\") " pod="openshift-dns/dns-default-m2fcv" Jul 07 15:43:32.189771 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:32.189771 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:43:32.192142 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:32.192109 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73fd3204-d75e-41e7-ae19-734ddf9d80ae-metrics-tls\") pod \"dns-default-m2fcv\" (UID: \"73fd3204-d75e-41e7-ae19-734ddf9d80ae\") " pod="openshift-dns/dns-default-m2fcv" Jul 07 15:43:32.192247 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:32.192177 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls\") pod \"image-registry-659cfff448-ggk7s\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:43:32.431273 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:32.431235 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-d8w2m\"" Jul 07 15:43:32.438609 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:32.438586 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:43:32.441308 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:32.441257 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kr7tb\"" Jul 07 15:43:32.449517 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:32.449494 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m2fcv" Jul 07 15:43:32.579304 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:32.579280 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-659cfff448-ggk7s"] Jul 07 15:43:32.581200 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:43:32.581176 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffe49307_5a6f_4ba2_9fa7_f28af223ab94.slice/crio-6478fcc3863b943fd97dceaf577b89bd9755f5185314b150758c9eaac6a85dfe WatchSource:0}: Error finding container 6478fcc3863b943fd97dceaf577b89bd9755f5185314b150758c9eaac6a85dfe: Status 404 returned error can't find the container with id 6478fcc3863b943fd97dceaf577b89bd9755f5185314b150758c9eaac6a85dfe Jul 07 15:43:32.594263 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:32.594241 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m2fcv"] Jul 07 15:43:32.597450 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:43:32.597428 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73fd3204_d75e_41e7_ae19_734ddf9d80ae.slice/crio-b2e10102284f2103b954b787d6b65bb009b9957fc49f0dd22abd19b6f974a014 WatchSource:0}: Error finding container b2e10102284f2103b954b787d6b65bb009b9957fc49f0dd22abd19b6f974a014: Status 404 returned error can't find the container with id b2e10102284f2103b954b787d6b65bb009b9957fc49f0dd22abd19b6f974a014 Jul 07 15:43:32.732359 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:32.732273 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fwrrt" Jul 07 15:43:32.844158 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:32.844022 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-2cld9" event={"ID":"0fc8fb5d-775b-4cf5-929d-21354c456e1b","Type":"ContainerStarted","Data":"2198902805b25cfe79c8c0a619e60eb8058f0cdc446a6243967332ec942e83fa"} Jul 07 15:43:32.845276 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:32.845238 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m2fcv" event={"ID":"73fd3204-d75e-41e7-ae19-734ddf9d80ae","Type":"ContainerStarted","Data":"b2e10102284f2103b954b787d6b65bb009b9957fc49f0dd22abd19b6f974a014"} Jul 07 15:43:32.847049 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:32.847027 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-659cfff448-ggk7s" event={"ID":"ffe49307-5a6f-4ba2-9fa7-f28af223ab94","Type":"ContainerStarted","Data":"94560bd3dd57f607f00e486ab7f874edc6a00a59333f3c1964a9dd2472d41702"} Jul 07 15:43:32.847151 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:32.847067 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-659cfff448-ggk7s" event={"ID":"ffe49307-5a6f-4ba2-9fa7-f28af223ab94","Type":"ContainerStarted","Data":"6478fcc3863b943fd97dceaf577b89bd9755f5185314b150758c9eaac6a85dfe"} Jul 07 15:43:32.847205 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:32.847151 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:43:32.864261 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:32.864218 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-2cld9" podStartSLOduration=2.246730661 podStartE2EDuration="3.864203124s" totalImagesPullingTime="1.617472463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:43:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:43:32.863344869 +0000 UTC m=+99.958625950" watchObservedRunningTime="2026-07-07 15:43:32.864203124 +0000 UTC m=+99.959484203" Jul 07 15:43:32.889738 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:32.889679 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-659cfff448-ggk7s" podStartSLOduration=98.889661853 podStartE2EDuration="1m38.889661853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:41:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:43:32.888787078 +0000 UTC m=+99.984068156" watchObservedRunningTime="2026-07-07 15:43:32.889661853 +0000 UTC m=+99.984942932" Jul 07 15:43:34.855088 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:34.855053 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m2fcv" event={"ID":"73fd3204-d75e-41e7-ae19-734ddf9d80ae","Type":"ContainerStarted","Data":"4c1d4dcbfcbb2b12cb75cb1f91bf93800b7416ca59536d47d335e5067476ec12"} Jul 07 15:43:34.855467 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:34.855135 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m2fcv" event={"ID":"73fd3204-d75e-41e7-ae19-734ddf9d80ae","Type":"ContainerStarted","Data":"0d38c12e7ef66424221a4326d3a940697359e051e3aa464304ede89d2a2369ae"} Jul 07 15:43:34.855467 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:34.855160 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-m2fcv" Jul 07 15:43:34.898312 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:34.898260 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-m2fcv" podStartSLOduration=65.402011823 podStartE2EDuration="1m6.898243937s" totalImagesPullingTime="1.496232114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:42:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:43:34.897146005 +0000 UTC m=+101.992427096" watchObservedRunningTime="2026-07-07 15:43:34.898243937 +0000 UTC m=+101.993525031" Jul 07 15:43:35.168185 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.168093 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6vp4d"] Jul 07 15:43:35.170963 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.170942 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.174542 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.174525 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 15:43:35.174944 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.174928 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 15:43:35.175266 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.175251 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 15:43:35.175496 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.175485 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-56vcq\"" Jul 07 15:43:35.317106 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.317075 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b42f19d6-c809-45c9-bc19-0d48b7e7b355-root\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.317303 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.317118 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b42f19d6-c809-45c9-bc19-0d48b7e7b355-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.317303 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.317140 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b42f19d6-c809-45c9-bc19-0d48b7e7b355-node-exporter-textfile\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.317303 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.317217 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b42f19d6-c809-45c9-bc19-0d48b7e7b355-sys\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.317303 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.317236 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b42f19d6-c809-45c9-bc19-0d48b7e7b355-node-exporter-tls\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.317303 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.317256 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b42f19d6-c809-45c9-bc19-0d48b7e7b355-node-exporter-wtmp\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.317303 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.317273 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b42f19d6-c809-45c9-bc19-0d48b7e7b355-node-exporter-accelerators-collector-config\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.317504 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.317342 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s24gs\" (UniqueName: \"kubernetes.io/projected/b42f19d6-c809-45c9-bc19-0d48b7e7b355-kube-api-access-s24gs\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.317504 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.317400 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b42f19d6-c809-45c9-bc19-0d48b7e7b355-metrics-client-ca\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.418101 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.418071 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b42f19d6-c809-45c9-bc19-0d48b7e7b355-sys\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.418101 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.418104 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b42f19d6-c809-45c9-bc19-0d48b7e7b355-node-exporter-tls\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.418374 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.418124 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b42f19d6-c809-45c9-bc19-0d48b7e7b355-node-exporter-wtmp\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.418374 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.418141 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b42f19d6-c809-45c9-bc19-0d48b7e7b355-node-exporter-accelerators-collector-config\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.418374 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.418162 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s24gs\" (UniqueName: \"kubernetes.io/projected/b42f19d6-c809-45c9-bc19-0d48b7e7b355-kube-api-access-s24gs\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.418374 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.418199 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b42f19d6-c809-45c9-bc19-0d48b7e7b355-metrics-client-ca\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.418374 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.418198 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b42f19d6-c809-45c9-bc19-0d48b7e7b355-sys\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.418374 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.418235 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b42f19d6-c809-45c9-bc19-0d48b7e7b355-root\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.418374 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.418268 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b42f19d6-c809-45c9-bc19-0d48b7e7b355-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.418374 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.418292 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b42f19d6-c809-45c9-bc19-0d48b7e7b355-node-exporter-wtmp\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.418374 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.418312 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b42f19d6-c809-45c9-bc19-0d48b7e7b355-root\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.418374 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.418300 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b42f19d6-c809-45c9-bc19-0d48b7e7b355-node-exporter-textfile\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.418712 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.418674 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b42f19d6-c809-45c9-bc19-0d48b7e7b355-node-exporter-textfile\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.418850 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.418829 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b42f19d6-c809-45c9-bc19-0d48b7e7b355-node-exporter-accelerators-collector-config\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.418906 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.418836 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b42f19d6-c809-45c9-bc19-0d48b7e7b355-metrics-client-ca\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.420607 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.420581 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b42f19d6-c809-45c9-bc19-0d48b7e7b355-node-exporter-tls\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.420711 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.420584 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b42f19d6-c809-45c9-bc19-0d48b7e7b355-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.433055 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.433025 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s24gs\" (UniqueName: \"kubernetes.io/projected/b42f19d6-c809-45c9-bc19-0d48b7e7b355-kube-api-access-s24gs\") pod \"node-exporter-6vp4d\" (UID: \"b42f19d6-c809-45c9-bc19-0d48b7e7b355\") " pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.479822 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.479793 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6vp4d" Jul 07 15:43:35.489734 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:43:35.489708 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb42f19d6_c809_45c9_bc19_0d48b7e7b355.slice/crio-343e49326e4a92fb4a2b50a14e8b5b3a235bb096609c4c4b281489e09e44f7ba WatchSource:0}: Error finding container 343e49326e4a92fb4a2b50a14e8b5b3a235bb096609c4c4b281489e09e44f7ba: Status 404 returned error can't find the container with id 343e49326e4a92fb4a2b50a14e8b5b3a235bb096609c4c4b281489e09e44f7ba Jul 07 15:43:35.859573 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:35.859538 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6vp4d" event={"ID":"b42f19d6-c809-45c9-bc19-0d48b7e7b355","Type":"ContainerStarted","Data":"343e49326e4a92fb4a2b50a14e8b5b3a235bb096609c4c4b281489e09e44f7ba"} Jul 07 15:43:36.270634 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.270546 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:43:36.292296 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.292264 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.296358 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.296170 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:43:36.296600 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.296577 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 07 15:43:36.296729 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.296615 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 07 15:43:36.296907 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.296826 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 07 15:43:36.297012 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.296965 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 07 15:43:36.297186 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.297170 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 07 15:43:36.298370 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.297888 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 07 15:43:36.298370 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.297920 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 07 15:43:36.298370 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.297928 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 07 15:43:36.298370 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.298171 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-bgxmq\"" Jul 07 15:43:36.310467 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.310445 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 07 15:43:36.425666 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.425635 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.425857 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.425680 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.425857 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.425722 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.425857 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.425778 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.425857 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.425825 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-config-volume\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.426031 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.425881 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.426031 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.425908 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.426031 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.425941 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.426169 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.426020 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkkfn\" (UniqueName: \"kubernetes.io/projected/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-kube-api-access-qkkfn\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.426220 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.426182 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-config-out\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.426257 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.426221 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.426299 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.426270 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.426381 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.426365 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-web-config\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.527254 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.527172 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkkfn\" (UniqueName: \"kubernetes.io/projected/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-kube-api-access-qkkfn\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.527254 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.527236 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-config-out\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.527451 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.527263 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.527451 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.527280 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.527451 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.527330 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-web-config\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.527451 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.527362 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.527451 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.527379 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.527451 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.527398 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.527451 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.527416 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.527451 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.527441 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-config-volume\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.527805 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.527477 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.527805 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.527501 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.527805 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.527529 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.529516 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.528114 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.529516 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:36.528240 2575 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 07 15:43:36.529516 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:43:36.528310 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-main-tls podName:2f42b18e-7bcf-4bf9-bf69-63d6d12b052e nodeName:}" failed. No retries permitted until 2026-07-07 15:43:37.028292239 +0000 UTC m=+104.123573311 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e") : secret "alertmanager-main-tls" not found Jul 07 15:43:36.529516 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.528480 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.529516 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.529161 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.530169 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.530123 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-config-out\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.530424 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.530378 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.530503 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.530480 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.530690 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.530655 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-config-volume\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.530828 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.530807 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-web-config\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.531099 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.531077 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.531153 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.531096 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.531787 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.531764 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.548125 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.548102 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkkfn\" (UniqueName: \"kubernetes.io/projected/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-kube-api-access-qkkfn\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:36.864803 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:36.864769 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6vp4d" event={"ID":"b42f19d6-c809-45c9-bc19-0d48b7e7b355","Type":"ContainerStarted","Data":"62c870f8d15782860667347c5fa2ce32561e4f922d93e09ca5ce067e0cf972dd"} Jul 07 15:43:37.031627 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.031585 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:37.034371 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.034335 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:37.211294 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.211258 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:43:37.243322 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.243294 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-564d665bfd-zdgh9"] Jul 07 15:43:37.246590 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.246571 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.249822 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.249396 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 07 15:43:37.249822 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.249422 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-vwltz\"" Jul 07 15:43:37.249822 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.249671 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 07 15:43:37.250396 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.250064 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 07 15:43:37.250396 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.250143 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 07 15:43:37.250396 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.250276 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 07 15:43:37.250396 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.250390 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-3cckt2uvld36m\"" Jul 07 15:43:37.263935 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.263907 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-564d665bfd-zdgh9"] Jul 07 15:43:37.334761 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.334727 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/02a7ff49-005b-4e1f-b21a-992d9828c8d8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-564d665bfd-zdgh9\" (UID: \"02a7ff49-005b-4e1f-b21a-992d9828c8d8\") " pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.334929 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.334783 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/02a7ff49-005b-4e1f-b21a-992d9828c8d8-secret-thanos-querier-tls\") pod \"thanos-querier-564d665bfd-zdgh9\" (UID: \"02a7ff49-005b-4e1f-b21a-992d9828c8d8\") " pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.334929 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.334894 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02a7ff49-005b-4e1f-b21a-992d9828c8d8-metrics-client-ca\") pod \"thanos-querier-564d665bfd-zdgh9\" (UID: \"02a7ff49-005b-4e1f-b21a-992d9828c8d8\") " pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.334999 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.334929 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/02a7ff49-005b-4e1f-b21a-992d9828c8d8-secret-grpc-tls\") pod \"thanos-querier-564d665bfd-zdgh9\" (UID: \"02a7ff49-005b-4e1f-b21a-992d9828c8d8\") " pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.334999 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.334967 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/02a7ff49-005b-4e1f-b21a-992d9828c8d8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-564d665bfd-zdgh9\" (UID: \"02a7ff49-005b-4e1f-b21a-992d9828c8d8\") " pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.334999 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.334987 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/02a7ff49-005b-4e1f-b21a-992d9828c8d8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-564d665bfd-zdgh9\" (UID: \"02a7ff49-005b-4e1f-b21a-992d9828c8d8\") " pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.335087 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.335008 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/02a7ff49-005b-4e1f-b21a-992d9828c8d8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-564d665bfd-zdgh9\" (UID: \"02a7ff49-005b-4e1f-b21a-992d9828c8d8\") " pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.335087 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.335070 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xc42\" (UniqueName: \"kubernetes.io/projected/02a7ff49-005b-4e1f-b21a-992d9828c8d8-kube-api-access-7xc42\") pod \"thanos-querier-564d665bfd-zdgh9\" (UID: \"02a7ff49-005b-4e1f-b21a-992d9828c8d8\") " pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.355176 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.355148 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:43:37.356459 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:43:37.356438 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f42b18e_7bcf_4bf9_bf69_63d6d12b052e.slice/crio-2393e6b141e7eb0c9f204c6abbe28ba46663889d0283ff136c3a191d7f3d7226 WatchSource:0}: Error finding container 2393e6b141e7eb0c9f204c6abbe28ba46663889d0283ff136c3a191d7f3d7226: Status 404 returned error can't find the container with id 2393e6b141e7eb0c9f204c6abbe28ba46663889d0283ff136c3a191d7f3d7226 Jul 07 15:43:37.436110 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.436083 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/02a7ff49-005b-4e1f-b21a-992d9828c8d8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-564d665bfd-zdgh9\" (UID: \"02a7ff49-005b-4e1f-b21a-992d9828c8d8\") " pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.436274 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.436125 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/02a7ff49-005b-4e1f-b21a-992d9828c8d8-secret-thanos-querier-tls\") pod \"thanos-querier-564d665bfd-zdgh9\" (UID: \"02a7ff49-005b-4e1f-b21a-992d9828c8d8\") " pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.436274 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.436177 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02a7ff49-005b-4e1f-b21a-992d9828c8d8-metrics-client-ca\") pod \"thanos-querier-564d665bfd-zdgh9\" (UID: \"02a7ff49-005b-4e1f-b21a-992d9828c8d8\") " pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.436274 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.436203 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/02a7ff49-005b-4e1f-b21a-992d9828c8d8-secret-grpc-tls\") pod \"thanos-querier-564d665bfd-zdgh9\" (UID: \"02a7ff49-005b-4e1f-b21a-992d9828c8d8\") " pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.436274 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.436238 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/02a7ff49-005b-4e1f-b21a-992d9828c8d8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-564d665bfd-zdgh9\" (UID: \"02a7ff49-005b-4e1f-b21a-992d9828c8d8\") " pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.436274 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.436266 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/02a7ff49-005b-4e1f-b21a-992d9828c8d8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-564d665bfd-zdgh9\" (UID: \"02a7ff49-005b-4e1f-b21a-992d9828c8d8\") " pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.436580 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.436296 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/02a7ff49-005b-4e1f-b21a-992d9828c8d8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-564d665bfd-zdgh9\" (UID: \"02a7ff49-005b-4e1f-b21a-992d9828c8d8\") " pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.436580 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.436327 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xc42\" (UniqueName: \"kubernetes.io/projected/02a7ff49-005b-4e1f-b21a-992d9828c8d8-kube-api-access-7xc42\") pod \"thanos-querier-564d665bfd-zdgh9\" (UID: \"02a7ff49-005b-4e1f-b21a-992d9828c8d8\") " pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.437024 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.436996 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02a7ff49-005b-4e1f-b21a-992d9828c8d8-metrics-client-ca\") pod \"thanos-querier-564d665bfd-zdgh9\" (UID: \"02a7ff49-005b-4e1f-b21a-992d9828c8d8\") " pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.438773 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.438752 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/02a7ff49-005b-4e1f-b21a-992d9828c8d8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-564d665bfd-zdgh9\" (UID: \"02a7ff49-005b-4e1f-b21a-992d9828c8d8\") " pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.438888 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.438837 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/02a7ff49-005b-4e1f-b21a-992d9828c8d8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-564d665bfd-zdgh9\" (UID: \"02a7ff49-005b-4e1f-b21a-992d9828c8d8\") " pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.439468 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.439447 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/02a7ff49-005b-4e1f-b21a-992d9828c8d8-secret-grpc-tls\") pod \"thanos-querier-564d665bfd-zdgh9\" (UID: \"02a7ff49-005b-4e1f-b21a-992d9828c8d8\") " pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.439538 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.439452 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/02a7ff49-005b-4e1f-b21a-992d9828c8d8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-564d665bfd-zdgh9\" (UID: \"02a7ff49-005b-4e1f-b21a-992d9828c8d8\") " pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.439538 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.439489 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/02a7ff49-005b-4e1f-b21a-992d9828c8d8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-564d665bfd-zdgh9\" (UID: \"02a7ff49-005b-4e1f-b21a-992d9828c8d8\") " pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.439617 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.439568 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/02a7ff49-005b-4e1f-b21a-992d9828c8d8-secret-thanos-querier-tls\") pod \"thanos-querier-564d665bfd-zdgh9\" (UID: \"02a7ff49-005b-4e1f-b21a-992d9828c8d8\") " pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.455556 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.455530 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xc42\" (UniqueName: \"kubernetes.io/projected/02a7ff49-005b-4e1f-b21a-992d9828c8d8-kube-api-access-7xc42\") pod \"thanos-querier-564d665bfd-zdgh9\" (UID: \"02a7ff49-005b-4e1f-b21a-992d9828c8d8\") " pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.556750 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.556672 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:37.696730 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.696696 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-564d665bfd-zdgh9"] Jul 07 15:43:37.699660 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:43:37.699626 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02a7ff49_005b_4e1f_b21a_992d9828c8d8.slice/crio-9b04b7451beaf14c8da1c2bba750d73186dcfae8ef3109d16e7e40e8d4b60cba WatchSource:0}: Error finding container 9b04b7451beaf14c8da1c2bba750d73186dcfae8ef3109d16e7e40e8d4b60cba: Status 404 returned error can't find the container with id 9b04b7451beaf14c8da1c2bba750d73186dcfae8ef3109d16e7e40e8d4b60cba Jul 07 15:43:37.869663 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.869632 2575 generic.go:356] "Generic (PLEG): container finished" podID="b42f19d6-c809-45c9-bc19-0d48b7e7b355" containerID="62c870f8d15782860667347c5fa2ce32561e4f922d93e09ca5ce067e0cf972dd" exitCode=0 Jul 07 15:43:37.870140 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.869728 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6vp4d" event={"ID":"b42f19d6-c809-45c9-bc19-0d48b7e7b355","Type":"ContainerDied","Data":"62c870f8d15782860667347c5fa2ce32561e4f922d93e09ca5ce067e0cf972dd"} Jul 07 15:43:37.870996 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.870971 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e","Type":"ContainerStarted","Data":"2393e6b141e7eb0c9f204c6abbe28ba46663889d0283ff136c3a191d7f3d7226"} Jul 07 15:43:37.871924 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:37.871904 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" event={"ID":"02a7ff49-005b-4e1f-b21a-992d9828c8d8","Type":"ContainerStarted","Data":"9b04b7451beaf14c8da1c2bba750d73186dcfae8ef3109d16e7e40e8d4b60cba"} Jul 07 15:43:38.879366 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:38.879327 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6vp4d" event={"ID":"b42f19d6-c809-45c9-bc19-0d48b7e7b355","Type":"ContainerStarted","Data":"d776db836547863a4510b032f4ba7dd5fceef4f4d6bcbd18c66dce2f6664072f"} Jul 07 15:43:38.879845 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:38.879426 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6vp4d" event={"ID":"b42f19d6-c809-45c9-bc19-0d48b7e7b355","Type":"ContainerStarted","Data":"019cb4e1418ac6e9f1de4b28785b4af2efd5a3c58ab07b0c4ef28792b1d1f83a"} Jul 07 15:43:38.905169 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:38.905119 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6vp4d" podStartSLOduration=2.5697321950000003 podStartE2EDuration="3.90510229s" totalImagesPullingTime="1.269819907s" totalInitContainerRuntime="65.550188ms" isStatefulPod=true podCreationTimestamp="2026-07-07 15:43:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:43:38.903724351 +0000 UTC m=+105.999005430" watchObservedRunningTime="2026-07-07 15:43:38.90510229 +0000 UTC m=+106.000383368" Jul 07 15:43:39.596669 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.596637 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-54c5cdd484-sdptj"] Jul 07 15:43:39.599323 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.599295 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:43:39.602547 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.602518 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 07 15:43:39.603303 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.603278 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-dnwqv\"" Jul 07 15:43:39.603303 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.603296 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 15:43:39.603543 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.603525 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 07 15:43:39.605706 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.605671 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ct2dd929aj2ao\"" Jul 07 15:43:39.605831 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.605754 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 07 15:43:39.617295 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.617271 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-54c5cdd484-sdptj"] Jul 07 15:43:39.756476 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.756434 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjwgk\" (UniqueName: \"kubernetes.io/projected/dfc229c2-f83c-441a-be18-cfdb40cf9097-kube-api-access-mjwgk\") pod \"metrics-server-54c5cdd484-sdptj\" (UID: \"dfc229c2-f83c-441a-be18-cfdb40cf9097\") " pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:43:39.756667 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.756571 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dfc229c2-f83c-441a-be18-cfdb40cf9097-secret-metrics-server-tls\") pod \"metrics-server-54c5cdd484-sdptj\" (UID: \"dfc229c2-f83c-441a-be18-cfdb40cf9097\") " pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:43:39.756667 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.756615 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfc229c2-f83c-441a-be18-cfdb40cf9097-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-54c5cdd484-sdptj\" (UID: \"dfc229c2-f83c-441a-be18-cfdb40cf9097\") " pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:43:39.756667 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.756636 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dfc229c2-f83c-441a-be18-cfdb40cf9097-audit-log\") pod \"metrics-server-54c5cdd484-sdptj\" (UID: \"dfc229c2-f83c-441a-be18-cfdb40cf9097\") " pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:43:39.756784 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.756700 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dfc229c2-f83c-441a-be18-cfdb40cf9097-metrics-server-audit-profiles\") pod \"metrics-server-54c5cdd484-sdptj\" (UID: \"dfc229c2-f83c-441a-be18-cfdb40cf9097\") " pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:43:39.756784 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.756763 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dfc229c2-f83c-441a-be18-cfdb40cf9097-secret-metrics-server-client-certs\") pod \"metrics-server-54c5cdd484-sdptj\" (UID: \"dfc229c2-f83c-441a-be18-cfdb40cf9097\") " pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:43:39.756854 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.756809 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfc229c2-f83c-441a-be18-cfdb40cf9097-client-ca-bundle\") pod \"metrics-server-54c5cdd484-sdptj\" (UID: \"dfc229c2-f83c-441a-be18-cfdb40cf9097\") " pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:43:39.829955 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.829909 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-xr2zq"] Jul 07 15:43:39.832728 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.832696 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-xr2zq" Jul 07 15:43:39.835496 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.835431 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-kk5qq\"" Jul 07 15:43:39.835812 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.835788 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 07 15:43:39.843068 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.843025 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-xr2zq"] Jul 07 15:43:39.857380 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.857324 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfc229c2-f83c-441a-be18-cfdb40cf9097-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-54c5cdd484-sdptj\" (UID: \"dfc229c2-f83c-441a-be18-cfdb40cf9097\") " pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:43:39.857380 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.857355 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dfc229c2-f83c-441a-be18-cfdb40cf9097-audit-log\") pod \"metrics-server-54c5cdd484-sdptj\" (UID: \"dfc229c2-f83c-441a-be18-cfdb40cf9097\") " pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:43:39.857547 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.857384 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dfc229c2-f83c-441a-be18-cfdb40cf9097-metrics-server-audit-profiles\") pod \"metrics-server-54c5cdd484-sdptj\" (UID: \"dfc229c2-f83c-441a-be18-cfdb40cf9097\") " pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:43:39.857547 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.857531 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dfc229c2-f83c-441a-be18-cfdb40cf9097-secret-metrics-server-client-certs\") pod \"metrics-server-54c5cdd484-sdptj\" (UID: \"dfc229c2-f83c-441a-be18-cfdb40cf9097\") " pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:43:39.857652 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.857576 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfc229c2-f83c-441a-be18-cfdb40cf9097-client-ca-bundle\") pod \"metrics-server-54c5cdd484-sdptj\" (UID: \"dfc229c2-f83c-441a-be18-cfdb40cf9097\") " pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:43:39.857652 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.857619 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjwgk\" (UniqueName: \"kubernetes.io/projected/dfc229c2-f83c-441a-be18-cfdb40cf9097-kube-api-access-mjwgk\") pod \"metrics-server-54c5cdd484-sdptj\" (UID: \"dfc229c2-f83c-441a-be18-cfdb40cf9097\") " pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:43:39.857761 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.857711 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dfc229c2-f83c-441a-be18-cfdb40cf9097-audit-log\") pod \"metrics-server-54c5cdd484-sdptj\" (UID: \"dfc229c2-f83c-441a-be18-cfdb40cf9097\") " pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:43:39.857761 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.857723 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dfc229c2-f83c-441a-be18-cfdb40cf9097-secret-metrics-server-tls\") pod \"metrics-server-54c5cdd484-sdptj\" (UID: \"dfc229c2-f83c-441a-be18-cfdb40cf9097\") " pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:43:39.858155 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.858130 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfc229c2-f83c-441a-be18-cfdb40cf9097-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-54c5cdd484-sdptj\" (UID: \"dfc229c2-f83c-441a-be18-cfdb40cf9097\") " pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:43:39.858316 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.858298 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dfc229c2-f83c-441a-be18-cfdb40cf9097-metrics-server-audit-profiles\") pod \"metrics-server-54c5cdd484-sdptj\" (UID: \"dfc229c2-f83c-441a-be18-cfdb40cf9097\") " pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:43:39.859981 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.859961 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dfc229c2-f83c-441a-be18-cfdb40cf9097-secret-metrics-server-client-certs\") pod \"metrics-server-54c5cdd484-sdptj\" (UID: \"dfc229c2-f83c-441a-be18-cfdb40cf9097\") " pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:43:39.860381 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.860362 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfc229c2-f83c-441a-be18-cfdb40cf9097-client-ca-bundle\") pod \"metrics-server-54c5cdd484-sdptj\" (UID: \"dfc229c2-f83c-441a-be18-cfdb40cf9097\") " pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:43:39.860442 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.860393 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dfc229c2-f83c-441a-be18-cfdb40cf9097-secret-metrics-server-tls\") pod \"metrics-server-54c5cdd484-sdptj\" (UID: \"dfc229c2-f83c-441a-be18-cfdb40cf9097\") " pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:43:39.867459 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.867441 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjwgk\" (UniqueName: \"kubernetes.io/projected/dfc229c2-f83c-441a-be18-cfdb40cf9097-kube-api-access-mjwgk\") pod \"metrics-server-54c5cdd484-sdptj\" (UID: \"dfc229c2-f83c-441a-be18-cfdb40cf9097\") " pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:43:39.885266 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.885234 2575 generic.go:356] "Generic (PLEG): container finished" podID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerID="cb46dcd89ddb7f5fd9bf14973a23f44d65daff7917ebb4cb6a00a2d2dba39467" exitCode=0 Jul 07 15:43:39.885618 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.885280 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e","Type":"ContainerDied","Data":"cb46dcd89ddb7f5fd9bf14973a23f44d65daff7917ebb4cb6a00a2d2dba39467"} Jul 07 15:43:39.910183 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.910160 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:43:39.958411 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.958283 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63c46789-b6d5-4c23-823e-75fa094aaf47-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-xr2zq\" (UID: \"63c46789-b6d5-4c23-823e-75fa094aaf47\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-xr2zq" Jul 07 15:43:39.958411 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:39.958358 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsgm4\" (UniqueName: \"kubernetes.io/projected/63c46789-b6d5-4c23-823e-75fa094aaf47-kube-api-access-vsgm4\") pod \"monitoring-plugin-d5dcff7cf-xr2zq\" (UID: \"63c46789-b6d5-4c23-823e-75fa094aaf47\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-xr2zq" Jul 07 15:43:40.055535 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.055470 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-54c5cdd484-sdptj"] Jul 07 15:43:40.059478 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:43:40.059454 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfc229c2_f83c_441a_be18_cfdb40cf9097.slice/crio-ecabbabbda19a318febe1fc38b36c26dab5a91762d4cd05cd7ee960f4f0b626c WatchSource:0}: Error finding container ecabbabbda19a318febe1fc38b36c26dab5a91762d4cd05cd7ee960f4f0b626c: Status 404 returned error can't find the container with id ecabbabbda19a318febe1fc38b36c26dab5a91762d4cd05cd7ee960f4f0b626c Jul 07 15:43:40.060582 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.059674 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63c46789-b6d5-4c23-823e-75fa094aaf47-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-xr2zq\" (UID: \"63c46789-b6d5-4c23-823e-75fa094aaf47\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-xr2zq" Jul 07 15:43:40.060582 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.059771 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsgm4\" (UniqueName: \"kubernetes.io/projected/63c46789-b6d5-4c23-823e-75fa094aaf47-kube-api-access-vsgm4\") pod \"monitoring-plugin-d5dcff7cf-xr2zq\" (UID: \"63c46789-b6d5-4c23-823e-75fa094aaf47\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-xr2zq" Jul 07 15:43:40.064831 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.062975 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63c46789-b6d5-4c23-823e-75fa094aaf47-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-xr2zq\" (UID: \"63c46789-b6d5-4c23-823e-75fa094aaf47\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-xr2zq" Jul 07 15:43:40.077205 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.077180 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsgm4\" (UniqueName: \"kubernetes.io/projected/63c46789-b6d5-4c23-823e-75fa094aaf47-kube-api-access-vsgm4\") pod \"monitoring-plugin-d5dcff7cf-xr2zq\" (UID: \"63c46789-b6d5-4c23-823e-75fa094aaf47\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-xr2zq" Jul 07 15:43:40.145337 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.145260 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-xr2zq" Jul 07 15:43:40.285530 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.285499 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-xr2zq"] Jul 07 15:43:40.288856 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:43:40.288816 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63c46789_b6d5_4c23_823e_75fa094aaf47.slice/crio-2b1db33e4ae1360c3b3688ab224837eec752bd82a353a97af576dcf3de474273 WatchSource:0}: Error finding container 2b1db33e4ae1360c3b3688ab224837eec752bd82a353a97af576dcf3de474273: Status 404 returned error can't find the container with id 2b1db33e4ae1360c3b3688ab224837eec752bd82a353a97af576dcf3de474273 Jul 07 15:43:40.360854 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.360810 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5b8596d8bf-5bw58"] Jul 07 15:43:40.363723 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.363699 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.366612 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.366591 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 07 15:43:40.366890 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.366848 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 07 15:43:40.366890 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.366887 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-txrv7\"" Jul 07 15:43:40.366995 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.366903 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 07 15:43:40.367248 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.367236 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 07 15:43:40.367294 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.367260 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 07 15:43:40.372597 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.372574 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 07 15:43:40.377813 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.377791 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5b8596d8bf-5bw58"] Jul 07 15:43:40.462513 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.462415 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/cc7d21d4-173e-451b-810d-56e285861599-federate-client-tls\") pod \"telemeter-client-5b8596d8bf-5bw58\" (UID: \"cc7d21d4-173e-451b-810d-56e285861599\") " pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.462513 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.462464 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc7d21d4-173e-451b-810d-56e285861599-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5b8596d8bf-5bw58\" (UID: \"cc7d21d4-173e-451b-810d-56e285861599\") " pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.462720 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.462544 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cc7d21d4-173e-451b-810d-56e285861599-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5b8596d8bf-5bw58\" (UID: \"cc7d21d4-173e-451b-810d-56e285861599\") " pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.462720 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.462578 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc7d21d4-173e-451b-810d-56e285861599-serving-certs-ca-bundle\") pod \"telemeter-client-5b8596d8bf-5bw58\" (UID: \"cc7d21d4-173e-451b-810d-56e285861599\") " pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.462720 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.462599 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc7d21d4-173e-451b-810d-56e285861599-metrics-client-ca\") pod \"telemeter-client-5b8596d8bf-5bw58\" (UID: \"cc7d21d4-173e-451b-810d-56e285861599\") " pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.462720 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.462617 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/cc7d21d4-173e-451b-810d-56e285861599-telemeter-client-tls\") pod \"telemeter-client-5b8596d8bf-5bw58\" (UID: \"cc7d21d4-173e-451b-810d-56e285861599\") " pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.462720 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.462635 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/cc7d21d4-173e-451b-810d-56e285861599-secret-telemeter-client\") pod \"telemeter-client-5b8596d8bf-5bw58\" (UID: \"cc7d21d4-173e-451b-810d-56e285861599\") " pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.462969 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.462745 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8vgz\" (UniqueName: \"kubernetes.io/projected/cc7d21d4-173e-451b-810d-56e285861599-kube-api-access-b8vgz\") pod \"telemeter-client-5b8596d8bf-5bw58\" (UID: \"cc7d21d4-173e-451b-810d-56e285861599\") " pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.563892 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.563841 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc7d21d4-173e-451b-810d-56e285861599-serving-certs-ca-bundle\") pod \"telemeter-client-5b8596d8bf-5bw58\" (UID: \"cc7d21d4-173e-451b-810d-56e285861599\") " pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.563892 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.563896 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc7d21d4-173e-451b-810d-56e285861599-metrics-client-ca\") pod \"telemeter-client-5b8596d8bf-5bw58\" (UID: \"cc7d21d4-173e-451b-810d-56e285861599\") " pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.564114 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.563927 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/cc7d21d4-173e-451b-810d-56e285861599-telemeter-client-tls\") pod \"telemeter-client-5b8596d8bf-5bw58\" (UID: \"cc7d21d4-173e-451b-810d-56e285861599\") " pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.564114 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.563949 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/cc7d21d4-173e-451b-810d-56e285861599-secret-telemeter-client\") pod \"telemeter-client-5b8596d8bf-5bw58\" (UID: \"cc7d21d4-173e-451b-810d-56e285861599\") " pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.564114 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.563983 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8vgz\" (UniqueName: \"kubernetes.io/projected/cc7d21d4-173e-451b-810d-56e285861599-kube-api-access-b8vgz\") pod \"telemeter-client-5b8596d8bf-5bw58\" (UID: \"cc7d21d4-173e-451b-810d-56e285861599\") " pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.564114 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.564040 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/cc7d21d4-173e-451b-810d-56e285861599-federate-client-tls\") pod \"telemeter-client-5b8596d8bf-5bw58\" (UID: \"cc7d21d4-173e-451b-810d-56e285861599\") " pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.564114 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.564073 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc7d21d4-173e-451b-810d-56e285861599-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5b8596d8bf-5bw58\" (UID: \"cc7d21d4-173e-451b-810d-56e285861599\") " pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.564381 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.564146 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cc7d21d4-173e-451b-810d-56e285861599-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5b8596d8bf-5bw58\" (UID: \"cc7d21d4-173e-451b-810d-56e285861599\") " pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.565071 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.564685 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc7d21d4-173e-451b-810d-56e285861599-serving-certs-ca-bundle\") pod \"telemeter-client-5b8596d8bf-5bw58\" (UID: \"cc7d21d4-173e-451b-810d-56e285861599\") " pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.565071 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.564722 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc7d21d4-173e-451b-810d-56e285861599-metrics-client-ca\") pod \"telemeter-client-5b8596d8bf-5bw58\" (UID: \"cc7d21d4-173e-451b-810d-56e285861599\") " pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.565282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.565225 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc7d21d4-173e-451b-810d-56e285861599-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5b8596d8bf-5bw58\" (UID: \"cc7d21d4-173e-451b-810d-56e285861599\") " pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.566828 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.566797 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/cc7d21d4-173e-451b-810d-56e285861599-secret-telemeter-client\") pod \"telemeter-client-5b8596d8bf-5bw58\" (UID: \"cc7d21d4-173e-451b-810d-56e285861599\") " pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.567026 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.567005 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/cc7d21d4-173e-451b-810d-56e285861599-telemeter-client-tls\") pod \"telemeter-client-5b8596d8bf-5bw58\" (UID: \"cc7d21d4-173e-451b-810d-56e285861599\") " pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.567128 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.567043 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/cc7d21d4-173e-451b-810d-56e285861599-federate-client-tls\") pod \"telemeter-client-5b8596d8bf-5bw58\" (UID: \"cc7d21d4-173e-451b-810d-56e285861599\") " pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.567233 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.567214 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cc7d21d4-173e-451b-810d-56e285861599-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5b8596d8bf-5bw58\" (UID: \"cc7d21d4-173e-451b-810d-56e285861599\") " pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.575083 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.575062 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8vgz\" (UniqueName: \"kubernetes.io/projected/cc7d21d4-173e-451b-810d-56e285861599-kube-api-access-b8vgz\") pod \"telemeter-client-5b8596d8bf-5bw58\" (UID: \"cc7d21d4-173e-451b-810d-56e285861599\") " pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.673216 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.673171 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" Jul 07 15:43:40.849419 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.848903 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5b8596d8bf-5bw58"] Jul 07 15:43:40.891977 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.891892 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-xr2zq" event={"ID":"63c46789-b6d5-4c23-823e-75fa094aaf47","Type":"ContainerStarted","Data":"2b1db33e4ae1360c3b3688ab224837eec752bd82a353a97af576dcf3de474273"} Jul 07 15:43:40.896103 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.895890 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" event={"ID":"02a7ff49-005b-4e1f-b21a-992d9828c8d8","Type":"ContainerStarted","Data":"f517b81e9d8faaa8e6f27a870bcae937872b113bf7192fb2b1c50698dd358441"} Jul 07 15:43:40.896103 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.895974 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" event={"ID":"02a7ff49-005b-4e1f-b21a-992d9828c8d8","Type":"ContainerStarted","Data":"f7d787e7b60b12b8adfa23cd6d226bdf6eeb25c5e1c6638612cbf7de7eafce31"} Jul 07 15:43:40.896103 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.896015 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" event={"ID":"02a7ff49-005b-4e1f-b21a-992d9828c8d8","Type":"ContainerStarted","Data":"dbc0395e464e3bcd6e137a781f028ff8099ec11d75edd9800f41875469bf31ec"} Jul 07 15:43:40.898069 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:40.897980 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" event={"ID":"dfc229c2-f83c-441a-be18-cfdb40cf9097","Type":"ContainerStarted","Data":"ecabbabbda19a318febe1fc38b36c26dab5a91762d4cd05cd7ee960f4f0b626c"} Jul 07 15:43:40.957260 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:43:40.957214 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc7d21d4_173e_451b_810d_56e285861599.slice/crio-720c2252e88205e86c2ecc30e349836000b0670387f9f443e4e808114bdd2023 WatchSource:0}: Error finding container 720c2252e88205e86c2ecc30e349836000b0670387f9f443e4e808114bdd2023: Status 404 returned error can't find the container with id 720c2252e88205e86c2ecc30e349836000b0670387f9f443e4e808114bdd2023 Jul 07 15:43:41.510117 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.510078 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 15:43:41.513428 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.513401 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.518449 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.518407 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 15:43:41.519165 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.519069 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 15:43:41.519165 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.519132 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 15:43:41.519650 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.519627 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 15:43:41.519976 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.519958 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 15:43:41.519976 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.519968 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 15:43:41.520132 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.519986 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 15:43:41.520254 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.520235 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 15:43:41.520338 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.520318 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9r4x8\"" Jul 07 15:43:41.520338 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.520244 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 15:43:41.520672 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.520652 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 15:43:41.520830 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.520808 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-amr96bvb0c8m4\"" Jul 07 15:43:41.523441 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.520854 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 15:43:41.523441 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.521191 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 15:43:41.523441 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.521669 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 15:43:41.527298 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.527280 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 15:43:41.544591 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.544561 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 15:43:41.676946 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.676907 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.677244 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.676971 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-config-out\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.677244 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.677011 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.677244 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.677041 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.677244 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.677069 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.677244 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.677093 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.677244 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.677117 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.677244 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.677148 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-config\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.677244 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.677191 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.677606 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.677248 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.677606 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.677331 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.677606 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.677380 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.677606 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.677407 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.677606 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.677452 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.677606 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.677475 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.677606 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.677500 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-web-config\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.677606 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.677519 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjpfw\" (UniqueName: \"kubernetes.io/projected/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-kube-api-access-wjpfw\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.677606 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.677552 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.677606 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.677581 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.677606 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.677597 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.778148 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.778069 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.778148 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.778131 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-config-out\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.778358 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.778154 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.778358 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.778278 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.778358 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.778317 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.778358 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.778340 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.778563 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.778363 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.778563 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.778400 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-config\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.778563 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.778424 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.778563 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.778474 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.778563 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.778503 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.778563 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.778531 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.778563 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.778556 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.778907 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.778594 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.778907 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.778619 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.778907 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.778613 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.778907 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.778674 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-web-config\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.778907 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.778714 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjpfw\" (UniqueName: \"kubernetes.io/projected/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-kube-api-access-wjpfw\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.778907 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.778757 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.778907 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.778797 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.778907 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.778826 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.779290 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.779214 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.780668 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.779573 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.780668 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.779713 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.780668 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.780375 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.783068 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.783042 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.783949 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.783925 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.785887 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.784893 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.785887 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.785075 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-config-out\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.785887 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.785387 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-web-config\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.785887 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.785516 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.785887 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.785559 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-config\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.786181 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.785973 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.786181 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.786062 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.787055 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.787015 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.787055 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.787029 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.787312 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.787290 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.788310 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.788284 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.788907 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.788854 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.791522 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.791481 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjpfw\" (UniqueName: \"kubernetes.io/projected/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-kube-api-access-wjpfw\") pod \"prometheus-k8s-0\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.827537 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.827471 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:41.911115 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.910678 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" event={"ID":"02a7ff49-005b-4e1f-b21a-992d9828c8d8","Type":"ContainerStarted","Data":"24fda16d9c598932c5fcd049d011d63349de0b048285f5f36bd9b77912922da3"} Jul 07 15:43:41.911115 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.910810 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" event={"ID":"02a7ff49-005b-4e1f-b21a-992d9828c8d8","Type":"ContainerStarted","Data":"fec317235ddbe19af22e1e9cb407d7c1aa7a39bb96ecf8d59a855c9a16986c21"} Jul 07 15:43:41.911115 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.910849 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" event={"ID":"02a7ff49-005b-4e1f-b21a-992d9828c8d8","Type":"ContainerStarted","Data":"2fb1130bea7222f79dc0d5bbcc6bc29da30b389451886520018cb800fc714bfe"} Jul 07 15:43:41.911115 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.910924 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:41.912800 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.912737 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" event={"ID":"cc7d21d4-173e-451b-810d-56e285861599","Type":"ContainerStarted","Data":"720c2252e88205e86c2ecc30e349836000b0670387f9f443e4e808114bdd2023"} Jul 07 15:43:41.955076 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:41.954990 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" podStartSLOduration=1.8862390169999999 podStartE2EDuration="4.954974148s" totalImagesPullingTime="3.068735131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:43:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:43:41.952720203 +0000 UTC m=+109.048001285" watchObservedRunningTime="2026-07-07 15:43:41.954974148 +0000 UTC m=+109.050255227" Jul 07 15:43:42.751268 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:42.751165 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 15:43:42.752588 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:43:42.752558 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f2e0ea8_80c9_4b02_b39d_e36e074638b6.slice/crio-314d0b571764c249392e9a0929fd16ed10eae61cccfc2658e4e7adf1fe532f1a WatchSource:0}: Error finding container 314d0b571764c249392e9a0929fd16ed10eae61cccfc2658e4e7adf1fe532f1a: Status 404 returned error can't find the container with id 314d0b571764c249392e9a0929fd16ed10eae61cccfc2658e4e7adf1fe532f1a Jul 07 15:43:42.918976 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:42.918927 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f2e0ea8-80c9-4b02-b39d-e36e074638b6","Type":"ContainerStarted","Data":"4c297a31451ac07ffc7c43435e20aa8e6679aa10e841c9c9f17342dcbb81e4e1"} Jul 07 15:43:42.919417 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:42.919009 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f2e0ea8-80c9-4b02-b39d-e36e074638b6","Type":"ContainerStarted","Data":"314d0b571764c249392e9a0929fd16ed10eae61cccfc2658e4e7adf1fe532f1a"} Jul 07 15:43:42.921883 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:42.921838 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e","Type":"ContainerStarted","Data":"acac4fbc39257984ed073afe25cfb18b96f3bc3a9b0e2212ff664b0bd09b2947"} Jul 07 15:43:42.922027 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:42.921935 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e","Type":"ContainerStarted","Data":"80e79b90249c773b579b3e108a8313963464878316b58dd8bec0888546992e69"} Jul 07 15:43:42.922027 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:42.921982 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e","Type":"ContainerStarted","Data":"7204143838d63913f9ec24b0a597563211c69d400c812d450ef0303d356cfa5b"} Jul 07 15:43:42.923808 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:42.923776 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-xr2zq" event={"ID":"63c46789-b6d5-4c23-823e-75fa094aaf47","Type":"ContainerStarted","Data":"267473d3191a91aca8232fc329d88072a65a265287ff44fd407b5916b4d91429"} Jul 07 15:43:42.924003 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:42.923978 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-xr2zq" Jul 07 15:43:42.925666 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:42.925595 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" event={"ID":"dfc229c2-f83c-441a-be18-cfdb40cf9097","Type":"ContainerStarted","Data":"c345cce08f90c751b8d8035aba91363bfc47d3efcd4a4861da843531df71d4e3"} Jul 07 15:43:42.929292 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:42.929274 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-xr2zq" Jul 07 15:43:42.987626 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:42.987571 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" podStartSLOduration=1.486134383 podStartE2EDuration="3.987551767s" totalImagesPullingTime="2.501417384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:43:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:43:42.985926464 +0000 UTC m=+110.081207545" watchObservedRunningTime="2026-07-07 15:43:42.987551767 +0000 UTC m=+110.082832848" Jul 07 15:43:43.010118 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:43.010063 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-xr2zq" podStartSLOduration=1.734048518 podStartE2EDuration="4.010046563s" totalImagesPullingTime="2.275998045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:43:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:43:43.008121219 +0000 UTC m=+110.103402297" watchObservedRunningTime="2026-07-07 15:43:43.010046563 +0000 UTC m=+110.105327642" Jul 07 15:43:43.933023 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:43.932986 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e","Type":"ContainerStarted","Data":"d64dfc45d2a3551a9d31970dbd92b186f40562e6146235709090048fcc3c42f8"} Jul 07 15:43:43.933487 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:43.933064 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e","Type":"ContainerStarted","Data":"bc2cbe75874d28cfd907ba27f5981aef5850ba29c35c87427c28320abbb1c27b"} Jul 07 15:43:43.933487 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:43.933099 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e","Type":"ContainerStarted","Data":"d681f7f7be4a1fe4139b7bc1988f975e28baa1715c9ed73d84ff804b9aab7898"} Jul 07 15:43:43.935036 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:43.935005 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" event={"ID":"cc7d21d4-173e-451b-810d-56e285861599","Type":"ContainerStarted","Data":"54683b09a1dc3e1c3e841ca626364a858fe1fcf87765933403745e1b12dfb51e"} Jul 07 15:43:43.935171 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:43.935063 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" event={"ID":"cc7d21d4-173e-451b-810d-56e285861599","Type":"ContainerStarted","Data":"4ea82ad0103276de4c840d9fb5ab3e5adfe61fb277975c3cec24f9578c9281ce"} Jul 07 15:43:43.935171 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:43.935093 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" event={"ID":"cc7d21d4-173e-451b-810d-56e285861599","Type":"ContainerStarted","Data":"810bfc70d49716b90c5ff9a54c7015411886ab9b7c3135a1fe625217bdf61b3c"} Jul 07 15:43:43.936473 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:43.936447 2575 generic.go:356] "Generic (PLEG): container finished" podID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerID="4c297a31451ac07ffc7c43435e20aa8e6679aa10e841c9c9f17342dcbb81e4e1" exitCode=0 Jul 07 15:43:43.936590 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:43.936529 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f2e0ea8-80c9-4b02-b39d-e36e074638b6","Type":"ContainerDied","Data":"4c297a31451ac07ffc7c43435e20aa8e6679aa10e841c9c9f17342dcbb81e4e1"} Jul 07 15:43:43.975082 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:43.975027 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.5785059930000003 podStartE2EDuration="7.975011921s" totalImagesPullingTime="4.373936728s" totalInitContainerRuntime="22.5692ms" isStatefulPod=false podCreationTimestamp="2026-07-07 15:43:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:43:43.973416572 +0000 UTC m=+111.068697652" watchObservedRunningTime="2026-07-07 15:43:43.975011921 +0000 UTC m=+111.070292999" Jul 07 15:43:44.050903 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:44.047359 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5b8596d8bf-5bw58" podStartSLOduration=1.6617236979999999 podStartE2EDuration="4.047337448s" totalImagesPullingTime="2.38561375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:43:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:43:44.044536764 +0000 UTC m=+111.139817844" watchObservedRunningTime="2026-07-07 15:43:44.047337448 +0000 UTC m=+111.142618529" Jul 07 15:43:44.862922 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:44.862885 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-m2fcv" Jul 07 15:43:46.897394 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:46.897355 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-659cfff448-ggk7s"] Jul 07 15:43:47.932116 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:47.932078 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-564d665bfd-zdgh9" Jul 07 15:43:47.961134 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:47.961096 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f2e0ea8-80c9-4b02-b39d-e36e074638b6","Type":"ContainerStarted","Data":"5dd8e6df2be1edba1167a3e771e786ff6bb274c5eff86c4bb9e1fbdacff183e1"} Jul 07 15:43:47.961303 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:47.961167 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f2e0ea8-80c9-4b02-b39d-e36e074638b6","Type":"ContainerStarted","Data":"36a1e3a5c9b676ebfe11ef63be9dd15fc2fb008c911151e6bcbf653336170abe"} Jul 07 15:43:47.961303 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:47.961197 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f2e0ea8-80c9-4b02-b39d-e36e074638b6","Type":"ContainerStarted","Data":"2163add76e4cc393e705d244e7af20ef6980d02ac098eba0505062dc23ea7d8b"} Jul 07 15:43:47.961303 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:47.961229 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f2e0ea8-80c9-4b02-b39d-e36e074638b6","Type":"ContainerStarted","Data":"c56af119761f6576dd4d589bf69f425b9d1b31ce2989ac8b0a33b77b7c4427f0"} Jul 07 15:43:47.961303 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:47.961264 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f2e0ea8-80c9-4b02-b39d-e36e074638b6","Type":"ContainerStarted","Data":"994e1c027f2bb272e9080cd99e14c58a6d9f9d224ac1e8f45581c298066afc3c"} Jul 07 15:43:47.961498 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:47.961307 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f2e0ea8-80c9-4b02-b39d-e36e074638b6","Type":"ContainerStarted","Data":"a3d7515e0b0a774fa4220d5fa860a589e8a75e2953f17d428f71d00ce545f44a"} Jul 07 15:43:48.009189 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:48.009132 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.6008020050000002 podStartE2EDuration="7.00911511s" totalImagesPullingTime="3.386797874s" totalInitContainerRuntime="21.515231ms" isStatefulPod=false podCreationTimestamp="2026-07-07 15:43:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:43:48.00635902 +0000 UTC m=+115.101640100" watchObservedRunningTime="2026-07-07 15:43:48.00911511 +0000 UTC m=+115.104396189" Jul 07 15:43:51.828127 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:51.828093 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:43:56.903169 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:56.903140 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:43:59.910627 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:59.910595 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:43:59.911102 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:43:59.910659 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:44:03.182511 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:03.182466 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs\") pod \"network-metrics-daemon-6p7tj\" (UID: \"39adfd20-ec8f-4114-ae3e-f6abc9c340f3\") " pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:44:03.184921 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:03.184894 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39adfd20-ec8f-4114-ae3e-f6abc9c340f3-metrics-certs\") pod \"network-metrics-daemon-6p7tj\" (UID: \"39adfd20-ec8f-4114-ae3e-f6abc9c340f3\") " pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:44:03.245500 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:03.245465 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-88kw6\"" Jul 07 15:44:03.252855 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:03.252827 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6p7tj" Jul 07 15:44:03.407536 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:03.407490 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6p7tj"] Jul 07 15:44:03.409988 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:44:03.409962 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39adfd20_ec8f_4114_ae3e_f6abc9c340f3.slice/crio-aff32a1930415a9ac28f4e11f8f4a0fa829ea15f7cfa4df3114658c2f1f9138d WatchSource:0}: Error finding container aff32a1930415a9ac28f4e11f8f4a0fa829ea15f7cfa4df3114658c2f1f9138d: Status 404 returned error can't find the container with id aff32a1930415a9ac28f4e11f8f4a0fa829ea15f7cfa4df3114658c2f1f9138d Jul 07 15:44:04.035741 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:04.035705 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6p7tj" event={"ID":"39adfd20-ec8f-4114-ae3e-f6abc9c340f3","Type":"ContainerStarted","Data":"aff32a1930415a9ac28f4e11f8f4a0fa829ea15f7cfa4df3114658c2f1f9138d"} Jul 07 15:44:05.041403 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:05.041311 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6p7tj" event={"ID":"39adfd20-ec8f-4114-ae3e-f6abc9c340f3","Type":"ContainerStarted","Data":"902c9f041118855bbd9327a1d35001536645fd736149167b6e82a7cd4317aae2"} Jul 07 15:44:05.041403 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:05.041372 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6p7tj" event={"ID":"39adfd20-ec8f-4114-ae3e-f6abc9c340f3","Type":"ContainerStarted","Data":"8d5467acb98426aced2ef88a33484f02c221bf2fd8018a6aaa34e71c8b471502"} Jul 07 15:44:05.064612 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:05.064551 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6p7tj" podStartSLOduration=130.753366902 podStartE2EDuration="2m12.064533909s" totalImagesPullingTime="1.311167007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:05.063836716 +0000 UTC m=+132.159117795" watchObservedRunningTime="2026-07-07 15:44:05.064533909 +0000 UTC m=+132.159814989" Jul 07 15:44:11.920674 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:11.920626 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-659cfff448-ggk7s" podUID="ffe49307-5a6f-4ba2-9fa7-f28af223ab94" containerName="registry" containerID="cri-o://94560bd3dd57f607f00e486ab7f874edc6a00a59333f3c1964a9dd2472d41702" gracePeriod=30 Jul 07 15:44:12.071907 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.071852 2575 generic.go:356] "Generic (PLEG): container finished" podID="ffe49307-5a6f-4ba2-9fa7-f28af223ab94" containerID="94560bd3dd57f607f00e486ab7f874edc6a00a59333f3c1964a9dd2472d41702" exitCode=0 Jul 07 15:44:12.072059 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.071951 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-659cfff448-ggk7s" event={"ID":"ffe49307-5a6f-4ba2-9fa7-f28af223ab94","Type":"ContainerDied","Data":"94560bd3dd57f607f00e486ab7f874edc6a00a59333f3c1964a9dd2472d41702"} Jul 07 15:44:12.215252 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.215225 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:44:12.363360 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.363326 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-ca-trust-extracted\") pod \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " Jul 07 15:44:12.363559 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.363379 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfcr9\" (UniqueName: \"kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-kube-api-access-sfcr9\") pod \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " Jul 07 15:44:12.363559 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.363411 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-trusted-ca\") pod \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " Jul 07 15:44:12.363559 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.363432 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-image-registry-private-configuration\") pod \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " Jul 07 15:44:12.363559 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.363458 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-bound-sa-token\") pod \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " Jul 07 15:44:12.363559 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.363496 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-certificates\") pod \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " Jul 07 15:44:12.363559 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.363528 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls\") pod \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " Jul 07 15:44:12.363836 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.363576 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-installation-pull-secrets\") pod \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\" (UID: \"ffe49307-5a6f-4ba2-9fa7-f28af223ab94\") " Jul 07 15:44:12.363976 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.363906 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ffe49307-5a6f-4ba2-9fa7-f28af223ab94" (UID: "ffe49307-5a6f-4ba2-9fa7-f28af223ab94"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:44:12.364113 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.364052 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ffe49307-5a6f-4ba2-9fa7-f28af223ab94" (UID: "ffe49307-5a6f-4ba2-9fa7-f28af223ab94"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:44:12.366180 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.366136 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ffe49307-5a6f-4ba2-9fa7-f28af223ab94" (UID: "ffe49307-5a6f-4ba2-9fa7-f28af223ab94"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:44:12.366180 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.366147 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ffe49307-5a6f-4ba2-9fa7-f28af223ab94" (UID: "ffe49307-5a6f-4ba2-9fa7-f28af223ab94"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:44:12.366363 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.366185 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ffe49307-5a6f-4ba2-9fa7-f28af223ab94" (UID: "ffe49307-5a6f-4ba2-9fa7-f28af223ab94"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:44:12.366363 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.366198 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-kube-api-access-sfcr9" (OuterVolumeSpecName: "kube-api-access-sfcr9") pod "ffe49307-5a6f-4ba2-9fa7-f28af223ab94" (UID: "ffe49307-5a6f-4ba2-9fa7-f28af223ab94"). InnerVolumeSpecName "kube-api-access-sfcr9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:44:12.366363 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.366335 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ffe49307-5a6f-4ba2-9fa7-f28af223ab94" (UID: "ffe49307-5a6f-4ba2-9fa7-f28af223ab94"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:44:12.373558 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.373528 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ffe49307-5a6f-4ba2-9fa7-f28af223ab94" (UID: "ffe49307-5a6f-4ba2-9fa7-f28af223ab94"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:44:12.464504 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.464410 2575 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-ca-trust-extracted\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:44:12.464504 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.464442 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfcr9\" (UniqueName: \"kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-kube-api-access-sfcr9\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:44:12.464504 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.464455 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-trusted-ca\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:44:12.464504 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.464468 2575 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-image-registry-private-configuration\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:44:12.464504 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.464482 2575 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-bound-sa-token\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:44:12.464504 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.464494 2575 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-certificates\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:44:12.464504 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.464508 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-registry-tls\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:44:12.464829 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:12.464520 2575 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ffe49307-5a6f-4ba2-9fa7-f28af223ab94-installation-pull-secrets\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:44:13.077273 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:13.077241 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-659cfff448-ggk7s" event={"ID":"ffe49307-5a6f-4ba2-9fa7-f28af223ab94","Type":"ContainerDied","Data":"6478fcc3863b943fd97dceaf577b89bd9755f5185314b150758c9eaac6a85dfe"} Jul 07 15:44:13.077273 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:13.077274 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-659cfff448-ggk7s" Jul 07 15:44:13.077729 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:13.077303 2575 scope.go:117] "RemoveContainer" containerID="94560bd3dd57f607f00e486ab7f874edc6a00a59333f3c1964a9dd2472d41702" Jul 07 15:44:13.105889 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:13.103475 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-659cfff448-ggk7s"] Jul 07 15:44:13.109754 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:13.109726 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-659cfff448-ggk7s"] Jul 07 15:44:13.434834 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:13.434801 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffe49307-5a6f-4ba2-9fa7-f28af223ab94" path="/var/lib/kubelet/pods/ffe49307-5a6f-4ba2-9fa7-f28af223ab94/volumes" Jul 07 15:44:19.916183 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:19.916154 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:44:19.920064 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:19.920041 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-54c5cdd484-sdptj" Jul 07 15:44:41.828468 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:41.828436 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:44:41.847964 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:41.847940 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:44:42.206282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:42.206257 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:44:55.798169 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:55.798130 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:44:55.798614 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:55.798441 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="alertmanager" containerID="cri-o://7204143838d63913f9ec24b0a597563211c69d400c812d450ef0303d356cfa5b" gracePeriod=120 Jul 07 15:44:55.798614 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:55.798491 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="kube-rbac-proxy-metric" containerID="cri-o://bc2cbe75874d28cfd907ba27f5981aef5850ba29c35c87427c28320abbb1c27b" gracePeriod=120 Jul 07 15:44:55.798614 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:55.798529 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="prom-label-proxy" containerID="cri-o://d64dfc45d2a3551a9d31970dbd92b186f40562e6146235709090048fcc3c42f8" gracePeriod=120 Jul 07 15:44:55.798614 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:55.798590 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="config-reloader" containerID="cri-o://80e79b90249c773b579b3e108a8313963464878316b58dd8bec0888546992e69" gracePeriod=120 Jul 07 15:44:55.798823 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:55.798613 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="kube-rbac-proxy" containerID="cri-o://d681f7f7be4a1fe4139b7bc1988f975e28baa1715c9ed73d84ff804b9aab7898" gracePeriod=120 Jul 07 15:44:55.798823 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:55.798592 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="kube-rbac-proxy-web" containerID="cri-o://acac4fbc39257984ed073afe25cfb18b96f3bc3a9b0e2212ff664b0bd09b2947" gracePeriod=120 Jul 07 15:44:56.261930 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:56.261897 2575 generic.go:356] "Generic (PLEG): container finished" podID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerID="d64dfc45d2a3551a9d31970dbd92b186f40562e6146235709090048fcc3c42f8" exitCode=0 Jul 07 15:44:56.261930 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:56.261922 2575 generic.go:356] "Generic (PLEG): container finished" podID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerID="d681f7f7be4a1fe4139b7bc1988f975e28baa1715c9ed73d84ff804b9aab7898" exitCode=0 Jul 07 15:44:56.261930 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:56.261939 2575 generic.go:356] "Generic (PLEG): container finished" podID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerID="80e79b90249c773b579b3e108a8313963464878316b58dd8bec0888546992e69" exitCode=0 Jul 07 15:44:56.262229 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:56.261946 2575 generic.go:356] "Generic (PLEG): container finished" podID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerID="7204143838d63913f9ec24b0a597563211c69d400c812d450ef0303d356cfa5b" exitCode=0 Jul 07 15:44:56.262229 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:56.261969 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e","Type":"ContainerDied","Data":"d64dfc45d2a3551a9d31970dbd92b186f40562e6146235709090048fcc3c42f8"} Jul 07 15:44:56.262229 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:56.262082 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e","Type":"ContainerDied","Data":"d681f7f7be4a1fe4139b7bc1988f975e28baa1715c9ed73d84ff804b9aab7898"} Jul 07 15:44:56.262229 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:56.262123 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e","Type":"ContainerDied","Data":"80e79b90249c773b579b3e108a8313963464878316b58dd8bec0888546992e69"} Jul 07 15:44:56.262229 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:56.262163 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e","Type":"ContainerDied","Data":"7204143838d63913f9ec24b0a597563211c69d400c812d450ef0303d356cfa5b"} Jul 07 15:44:57.035557 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.035534 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.150507 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.150418 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " Jul 07 15:44:57.150507 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.150486 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-main-tls\") pod \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " Jul 07 15:44:57.150727 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.150610 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-kube-rbac-proxy-web\") pod \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " Jul 07 15:44:57.150727 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.150653 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkkfn\" (UniqueName: \"kubernetes.io/projected/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-kube-api-access-qkkfn\") pod \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " Jul 07 15:44:57.150727 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.150680 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-config-out\") pod \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " Jul 07 15:44:57.150727 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.150711 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-alertmanager-trusted-ca-bundle\") pod \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " Jul 07 15:44:57.150975 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.150750 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-web-config\") pod \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " Jul 07 15:44:57.150975 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.150795 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-metrics-client-ca\") pod \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " Jul 07 15:44:57.150975 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.150817 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-tls-assets\") pod \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " Jul 07 15:44:57.150975 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.150858 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-config-volume\") pod \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " Jul 07 15:44:57.150975 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.150915 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-kube-rbac-proxy\") pod \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " Jul 07 15:44:57.150975 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.150946 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-alertmanager-main-db\") pod \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " Jul 07 15:44:57.151258 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.150995 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-cluster-tls-config\") pod \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\" (UID: \"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e\") " Jul 07 15:44:57.152528 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.151464 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" (UID: "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:44:57.152528 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.152174 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" (UID: "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:44:57.152528 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.152477 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" (UID: "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:44:57.153555 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.153527 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" (UID: "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:44:57.153782 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.153743 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" (UID: "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:44:57.154279 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.154245 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" (UID: "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:44:57.154371 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.154300 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-config-out" (OuterVolumeSpecName: "config-out") pod "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" (UID: "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:44:57.154729 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.154703 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-kube-api-access-qkkfn" (OuterVolumeSpecName: "kube-api-access-qkkfn") pod "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" (UID: "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e"). InnerVolumeSpecName "kube-api-access-qkkfn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:44:57.155015 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.154980 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-config-volume" (OuterVolumeSpecName: "config-volume") pod "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" (UID: "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:44:57.155140 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.155123 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" (UID: "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:44:57.155413 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.155396 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" (UID: "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:44:57.158160 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.158137 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" (UID: "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:44:57.164720 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.164700 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-web-config" (OuterVolumeSpecName: "web-config") pod "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" (UID: "2f42b18e-7bcf-4bf9-bf69-63d6d12b052e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:44:57.252687 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.252649 2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:44:57.252687 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.252683 2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-alertmanager-main-db\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:44:57.252687 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.252693 2575 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-cluster-tls-config\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:44:57.252687 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.252702 2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:44:57.252962 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.252711 2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-main-tls\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:44:57.252962 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.252720 2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:44:57.252962 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.252728 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkkfn\" (UniqueName: \"kubernetes.io/projected/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-kube-api-access-qkkfn\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:44:57.252962 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.252737 2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-config-out\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:44:57.252962 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.252745 2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:44:57.252962 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.252754 2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-web-config\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:44:57.252962 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.252762 2575 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-metrics-client-ca\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:44:57.252962 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.252769 2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-tls-assets\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:44:57.252962 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.252777 2575 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e-config-volume\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:44:57.268740 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.268709 2575 generic.go:356] "Generic (PLEG): container finished" podID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerID="bc2cbe75874d28cfd907ba27f5981aef5850ba29c35c87427c28320abbb1c27b" exitCode=0 Jul 07 15:44:57.268740 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.268734 2575 generic.go:356] "Generic (PLEG): container finished" podID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerID="acac4fbc39257984ed073afe25cfb18b96f3bc3a9b0e2212ff664b0bd09b2947" exitCode=0 Jul 07 15:44:57.268951 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.268784 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.268951 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.268778 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e","Type":"ContainerDied","Data":"bc2cbe75874d28cfd907ba27f5981aef5850ba29c35c87427c28320abbb1c27b"} Jul 07 15:44:57.269058 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.268953 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e","Type":"ContainerDied","Data":"acac4fbc39257984ed073afe25cfb18b96f3bc3a9b0e2212ff664b0bd09b2947"} Jul 07 15:44:57.269058 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.269003 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2f42b18e-7bcf-4bf9-bf69-63d6d12b052e","Type":"ContainerDied","Data":"2393e6b141e7eb0c9f204c6abbe28ba46663889d0283ff136c3a191d7f3d7226"} Jul 07 15:44:57.269058 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.269010 2575 scope.go:117] "RemoveContainer" containerID="d64dfc45d2a3551a9d31970dbd92b186f40562e6146235709090048fcc3c42f8" Jul 07 15:44:57.277189 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.277173 2575 scope.go:117] "RemoveContainer" containerID="bc2cbe75874d28cfd907ba27f5981aef5850ba29c35c87427c28320abbb1c27b" Jul 07 15:44:57.284374 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.284354 2575 scope.go:117] "RemoveContainer" containerID="d681f7f7be4a1fe4139b7bc1988f975e28baa1715c9ed73d84ff804b9aab7898" Jul 07 15:44:57.290887 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.290858 2575 scope.go:117] "RemoveContainer" containerID="acac4fbc39257984ed073afe25cfb18b96f3bc3a9b0e2212ff664b0bd09b2947" Jul 07 15:44:57.297033 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.297010 2575 scope.go:117] "RemoveContainer" containerID="80e79b90249c773b579b3e108a8313963464878316b58dd8bec0888546992e69" Jul 07 15:44:57.301106 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.301064 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:44:57.304306 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.304290 2575 scope.go:117] "RemoveContainer" containerID="7204143838d63913f9ec24b0a597563211c69d400c812d450ef0303d356cfa5b" Jul 07 15:44:57.307714 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.307690 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:44:57.311222 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.311204 2575 scope.go:117] "RemoveContainer" containerID="cb46dcd89ddb7f5fd9bf14973a23f44d65daff7917ebb4cb6a00a2d2dba39467" Jul 07 15:44:57.317930 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.317913 2575 scope.go:117] "RemoveContainer" containerID="d64dfc45d2a3551a9d31970dbd92b186f40562e6146235709090048fcc3c42f8" Jul 07 15:44:57.318200 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:44:57.318177 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d64dfc45d2a3551a9d31970dbd92b186f40562e6146235709090048fcc3c42f8\": container with ID starting with d64dfc45d2a3551a9d31970dbd92b186f40562e6146235709090048fcc3c42f8 not found: ID does not exist" containerID="d64dfc45d2a3551a9d31970dbd92b186f40562e6146235709090048fcc3c42f8" Jul 07 15:44:57.318257 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.318205 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d64dfc45d2a3551a9d31970dbd92b186f40562e6146235709090048fcc3c42f8"} err="failed to get container status \"d64dfc45d2a3551a9d31970dbd92b186f40562e6146235709090048fcc3c42f8\": rpc error: code = NotFound desc = could not find container \"d64dfc45d2a3551a9d31970dbd92b186f40562e6146235709090048fcc3c42f8\": container with ID starting with d64dfc45d2a3551a9d31970dbd92b186f40562e6146235709090048fcc3c42f8 not found: ID does not exist" Jul 07 15:44:57.318257 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.318239 2575 scope.go:117] "RemoveContainer" containerID="bc2cbe75874d28cfd907ba27f5981aef5850ba29c35c87427c28320abbb1c27b" Jul 07 15:44:57.318461 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:44:57.318439 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc2cbe75874d28cfd907ba27f5981aef5850ba29c35c87427c28320abbb1c27b\": container with ID starting with bc2cbe75874d28cfd907ba27f5981aef5850ba29c35c87427c28320abbb1c27b not found: ID does not exist" containerID="bc2cbe75874d28cfd907ba27f5981aef5850ba29c35c87427c28320abbb1c27b" Jul 07 15:44:57.318541 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.318463 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc2cbe75874d28cfd907ba27f5981aef5850ba29c35c87427c28320abbb1c27b"} err="failed to get container status \"bc2cbe75874d28cfd907ba27f5981aef5850ba29c35c87427c28320abbb1c27b\": rpc error: code = NotFound desc = could not find container \"bc2cbe75874d28cfd907ba27f5981aef5850ba29c35c87427c28320abbb1c27b\": container with ID starting with bc2cbe75874d28cfd907ba27f5981aef5850ba29c35c87427c28320abbb1c27b not found: ID does not exist" Jul 07 15:44:57.318541 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.318477 2575 scope.go:117] "RemoveContainer" containerID="d681f7f7be4a1fe4139b7bc1988f975e28baa1715c9ed73d84ff804b9aab7898" Jul 07 15:44:57.318682 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:44:57.318664 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d681f7f7be4a1fe4139b7bc1988f975e28baa1715c9ed73d84ff804b9aab7898\": container with ID starting with d681f7f7be4a1fe4139b7bc1988f975e28baa1715c9ed73d84ff804b9aab7898 not found: ID does not exist" containerID="d681f7f7be4a1fe4139b7bc1988f975e28baa1715c9ed73d84ff804b9aab7898" Jul 07 15:44:57.318720 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.318691 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d681f7f7be4a1fe4139b7bc1988f975e28baa1715c9ed73d84ff804b9aab7898"} err="failed to get container status \"d681f7f7be4a1fe4139b7bc1988f975e28baa1715c9ed73d84ff804b9aab7898\": rpc error: code = NotFound desc = could not find container \"d681f7f7be4a1fe4139b7bc1988f975e28baa1715c9ed73d84ff804b9aab7898\": container with ID starting with d681f7f7be4a1fe4139b7bc1988f975e28baa1715c9ed73d84ff804b9aab7898 not found: ID does not exist" Jul 07 15:44:57.318720 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.318707 2575 scope.go:117] "RemoveContainer" containerID="acac4fbc39257984ed073afe25cfb18b96f3bc3a9b0e2212ff664b0bd09b2947" Jul 07 15:44:57.318911 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:44:57.318896 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acac4fbc39257984ed073afe25cfb18b96f3bc3a9b0e2212ff664b0bd09b2947\": container with ID starting with acac4fbc39257984ed073afe25cfb18b96f3bc3a9b0e2212ff664b0bd09b2947 not found: ID does not exist" containerID="acac4fbc39257984ed073afe25cfb18b96f3bc3a9b0e2212ff664b0bd09b2947" Jul 07 15:44:57.318958 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.318915 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acac4fbc39257984ed073afe25cfb18b96f3bc3a9b0e2212ff664b0bd09b2947"} err="failed to get container status \"acac4fbc39257984ed073afe25cfb18b96f3bc3a9b0e2212ff664b0bd09b2947\": rpc error: code = NotFound desc = could not find container \"acac4fbc39257984ed073afe25cfb18b96f3bc3a9b0e2212ff664b0bd09b2947\": container with ID starting with acac4fbc39257984ed073afe25cfb18b96f3bc3a9b0e2212ff664b0bd09b2947 not found: ID does not exist" Jul 07 15:44:57.318958 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.318927 2575 scope.go:117] "RemoveContainer" containerID="80e79b90249c773b579b3e108a8313963464878316b58dd8bec0888546992e69" Jul 07 15:44:57.319124 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:44:57.319110 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80e79b90249c773b579b3e108a8313963464878316b58dd8bec0888546992e69\": container with ID starting with 80e79b90249c773b579b3e108a8313963464878316b58dd8bec0888546992e69 not found: ID does not exist" containerID="80e79b90249c773b579b3e108a8313963464878316b58dd8bec0888546992e69" Jul 07 15:44:57.319163 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.319127 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80e79b90249c773b579b3e108a8313963464878316b58dd8bec0888546992e69"} err="failed to get container status \"80e79b90249c773b579b3e108a8313963464878316b58dd8bec0888546992e69\": rpc error: code = NotFound desc = could not find container \"80e79b90249c773b579b3e108a8313963464878316b58dd8bec0888546992e69\": container with ID starting with 80e79b90249c773b579b3e108a8313963464878316b58dd8bec0888546992e69 not found: ID does not exist" Jul 07 15:44:57.319163 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.319141 2575 scope.go:117] "RemoveContainer" containerID="7204143838d63913f9ec24b0a597563211c69d400c812d450ef0303d356cfa5b" Jul 07 15:44:57.319336 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:44:57.319322 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7204143838d63913f9ec24b0a597563211c69d400c812d450ef0303d356cfa5b\": container with ID starting with 7204143838d63913f9ec24b0a597563211c69d400c812d450ef0303d356cfa5b not found: ID does not exist" containerID="7204143838d63913f9ec24b0a597563211c69d400c812d450ef0303d356cfa5b" Jul 07 15:44:57.319373 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.319339 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7204143838d63913f9ec24b0a597563211c69d400c812d450ef0303d356cfa5b"} err="failed to get container status \"7204143838d63913f9ec24b0a597563211c69d400c812d450ef0303d356cfa5b\": rpc error: code = NotFound desc = could not find container \"7204143838d63913f9ec24b0a597563211c69d400c812d450ef0303d356cfa5b\": container with ID starting with 7204143838d63913f9ec24b0a597563211c69d400c812d450ef0303d356cfa5b not found: ID does not exist" Jul 07 15:44:57.319373 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.319349 2575 scope.go:117] "RemoveContainer" containerID="cb46dcd89ddb7f5fd9bf14973a23f44d65daff7917ebb4cb6a00a2d2dba39467" Jul 07 15:44:57.319524 ip-10-0-131-221 kubenswrapper[2575]: E0707 15:44:57.319511 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb46dcd89ddb7f5fd9bf14973a23f44d65daff7917ebb4cb6a00a2d2dba39467\": container with ID starting with cb46dcd89ddb7f5fd9bf14973a23f44d65daff7917ebb4cb6a00a2d2dba39467 not found: ID does not exist" containerID="cb46dcd89ddb7f5fd9bf14973a23f44d65daff7917ebb4cb6a00a2d2dba39467" Jul 07 15:44:57.319559 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.319527 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb46dcd89ddb7f5fd9bf14973a23f44d65daff7917ebb4cb6a00a2d2dba39467"} err="failed to get container status \"cb46dcd89ddb7f5fd9bf14973a23f44d65daff7917ebb4cb6a00a2d2dba39467\": rpc error: code = NotFound desc = could not find container \"cb46dcd89ddb7f5fd9bf14973a23f44d65daff7917ebb4cb6a00a2d2dba39467\": container with ID starting with cb46dcd89ddb7f5fd9bf14973a23f44d65daff7917ebb4cb6a00a2d2dba39467 not found: ID does not exist" Jul 07 15:44:57.319559 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.319538 2575 scope.go:117] "RemoveContainer" containerID="d64dfc45d2a3551a9d31970dbd92b186f40562e6146235709090048fcc3c42f8" Jul 07 15:44:57.319724 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.319708 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d64dfc45d2a3551a9d31970dbd92b186f40562e6146235709090048fcc3c42f8"} err="failed to get container status \"d64dfc45d2a3551a9d31970dbd92b186f40562e6146235709090048fcc3c42f8\": rpc error: code = NotFound desc = could not find container \"d64dfc45d2a3551a9d31970dbd92b186f40562e6146235709090048fcc3c42f8\": container with ID starting with d64dfc45d2a3551a9d31970dbd92b186f40562e6146235709090048fcc3c42f8 not found: ID does not exist" Jul 07 15:44:57.319764 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.319724 2575 scope.go:117] "RemoveContainer" containerID="bc2cbe75874d28cfd907ba27f5981aef5850ba29c35c87427c28320abbb1c27b" Jul 07 15:44:57.319926 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.319902 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc2cbe75874d28cfd907ba27f5981aef5850ba29c35c87427c28320abbb1c27b"} err="failed to get container status \"bc2cbe75874d28cfd907ba27f5981aef5850ba29c35c87427c28320abbb1c27b\": rpc error: code = NotFound desc = could not find container \"bc2cbe75874d28cfd907ba27f5981aef5850ba29c35c87427c28320abbb1c27b\": container with ID starting with bc2cbe75874d28cfd907ba27f5981aef5850ba29c35c87427c28320abbb1c27b not found: ID does not exist" Jul 07 15:44:57.319976 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.319927 2575 scope.go:117] "RemoveContainer" containerID="d681f7f7be4a1fe4139b7bc1988f975e28baa1715c9ed73d84ff804b9aab7898" Jul 07 15:44:57.320122 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.320107 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d681f7f7be4a1fe4139b7bc1988f975e28baa1715c9ed73d84ff804b9aab7898"} err="failed to get container status \"d681f7f7be4a1fe4139b7bc1988f975e28baa1715c9ed73d84ff804b9aab7898\": rpc error: code = NotFound desc = could not find container \"d681f7f7be4a1fe4139b7bc1988f975e28baa1715c9ed73d84ff804b9aab7898\": container with ID starting with d681f7f7be4a1fe4139b7bc1988f975e28baa1715c9ed73d84ff804b9aab7898 not found: ID does not exist" Jul 07 15:44:57.320168 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.320122 2575 scope.go:117] "RemoveContainer" containerID="acac4fbc39257984ed073afe25cfb18b96f3bc3a9b0e2212ff664b0bd09b2947" Jul 07 15:44:57.320300 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.320284 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acac4fbc39257984ed073afe25cfb18b96f3bc3a9b0e2212ff664b0bd09b2947"} err="failed to get container status \"acac4fbc39257984ed073afe25cfb18b96f3bc3a9b0e2212ff664b0bd09b2947\": rpc error: code = NotFound desc = could not find container \"acac4fbc39257984ed073afe25cfb18b96f3bc3a9b0e2212ff664b0bd09b2947\": container with ID starting with acac4fbc39257984ed073afe25cfb18b96f3bc3a9b0e2212ff664b0bd09b2947 not found: ID does not exist" Jul 07 15:44:57.320341 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.320301 2575 scope.go:117] "RemoveContainer" containerID="80e79b90249c773b579b3e108a8313963464878316b58dd8bec0888546992e69" Jul 07 15:44:57.320476 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.320461 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80e79b90249c773b579b3e108a8313963464878316b58dd8bec0888546992e69"} err="failed to get container status \"80e79b90249c773b579b3e108a8313963464878316b58dd8bec0888546992e69\": rpc error: code = NotFound desc = could not find container \"80e79b90249c773b579b3e108a8313963464878316b58dd8bec0888546992e69\": container with ID starting with 80e79b90249c773b579b3e108a8313963464878316b58dd8bec0888546992e69 not found: ID does not exist" Jul 07 15:44:57.320521 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.320476 2575 scope.go:117] "RemoveContainer" containerID="7204143838d63913f9ec24b0a597563211c69d400c812d450ef0303d356cfa5b" Jul 07 15:44:57.320648 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.320633 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7204143838d63913f9ec24b0a597563211c69d400c812d450ef0303d356cfa5b"} err="failed to get container status \"7204143838d63913f9ec24b0a597563211c69d400c812d450ef0303d356cfa5b\": rpc error: code = NotFound desc = could not find container \"7204143838d63913f9ec24b0a597563211c69d400c812d450ef0303d356cfa5b\": container with ID starting with 7204143838d63913f9ec24b0a597563211c69d400c812d450ef0303d356cfa5b not found: ID does not exist" Jul 07 15:44:57.320696 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.320648 2575 scope.go:117] "RemoveContainer" containerID="cb46dcd89ddb7f5fd9bf14973a23f44d65daff7917ebb4cb6a00a2d2dba39467" Jul 07 15:44:57.320822 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.320808 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb46dcd89ddb7f5fd9bf14973a23f44d65daff7917ebb4cb6a00a2d2dba39467"} err="failed to get container status \"cb46dcd89ddb7f5fd9bf14973a23f44d65daff7917ebb4cb6a00a2d2dba39467\": rpc error: code = NotFound desc = could not find container \"cb46dcd89ddb7f5fd9bf14973a23f44d65daff7917ebb4cb6a00a2d2dba39467\": container with ID starting with cb46dcd89ddb7f5fd9bf14973a23f44d65daff7917ebb4cb6a00a2d2dba39467 not found: ID does not exist" Jul 07 15:44:57.341579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.341560 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:44:57.341858 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.341846 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffe49307-5a6f-4ba2-9fa7-f28af223ab94" containerName="registry" Jul 07 15:44:57.341858 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.341860 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffe49307-5a6f-4ba2-9fa7-f28af223ab94" containerName="registry" Jul 07 15:44:57.341955 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.341892 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="alertmanager" Jul 07 15:44:57.341955 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.341898 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="alertmanager" Jul 07 15:44:57.341955 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.341906 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="config-reloader" Jul 07 15:44:57.341955 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.341913 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="config-reloader" Jul 07 15:44:57.341955 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.341919 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="kube-rbac-proxy-web" Jul 07 15:44:57.341955 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.341924 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="kube-rbac-proxy-web" Jul 07 15:44:57.341955 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.341931 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="kube-rbac-proxy" Jul 07 15:44:57.341955 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.341937 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="kube-rbac-proxy" Jul 07 15:44:57.341955 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.341945 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="init-config-reloader" Jul 07 15:44:57.341955 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.341950 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="init-config-reloader" Jul 07 15:44:57.341955 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.341958 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="prom-label-proxy" Jul 07 15:44:57.342253 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.341964 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="prom-label-proxy" Jul 07 15:44:57.342253 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.341972 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="kube-rbac-proxy-metric" Jul 07 15:44:57.342253 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.341977 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="kube-rbac-proxy-metric" Jul 07 15:44:57.342253 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.342021 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="config-reloader" Jul 07 15:44:57.342253 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.342031 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="prom-label-proxy" Jul 07 15:44:57.342253 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.342039 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="kube-rbac-proxy-metric" Jul 07 15:44:57.342253 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.342044 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffe49307-5a6f-4ba2-9fa7-f28af223ab94" containerName="registry" Jul 07 15:44:57.342253 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.342050 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="alertmanager" Jul 07 15:44:57.342253 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.342054 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="kube-rbac-proxy-web" Jul 07 15:44:57.342253 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.342060 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" containerName="kube-rbac-proxy" Jul 07 15:44:57.345610 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.345594 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.348477 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.348461 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-bgxmq\"" Jul 07 15:44:57.348477 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.348468 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 07 15:44:57.348923 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.348910 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 07 15:44:57.349037 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.349021 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 07 15:44:57.349114 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.349027 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 07 15:44:57.349163 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.349151 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 07 15:44:57.349282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.349266 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 07 15:44:57.349401 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.349367 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 07 15:44:57.349551 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.349536 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 07 15:44:57.355722 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.355701 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 07 15:44:57.362573 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.362552 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:44:57.438952 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.438856 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f42b18e-7bcf-4bf9-bf69-63d6d12b052e" path="/var/lib/kubelet/pods/2f42b18e-7bcf-4bf9-bf69-63d6d12b052e/volumes" Jul 07 15:44:57.453736 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.453707 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.453736 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.453736 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.453944 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.453762 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-config-volume\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.453944 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.453779 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-web-config\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.453944 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.453843 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-config-out\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.453944 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.453927 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.454081 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.453953 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.454081 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.453980 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.454081 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.454001 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.454081 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.454030 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.454081 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.454051 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.454081 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.454070 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.454268 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.454088 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd9wn\" (UniqueName: \"kubernetes.io/projected/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-kube-api-access-kd9wn\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.555435 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.555400 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.555435 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.555442 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.555658 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.555574 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.555658 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.555616 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.555658 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.555647 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.555809 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.555695 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd9wn\" (UniqueName: \"kubernetes.io/projected/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-kube-api-access-kd9wn\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.555809 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.555730 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.555809 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.555756 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.555809 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.555795 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-config-volume\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.556054 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.555826 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-web-config\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.556054 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.555909 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-config-out\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.556054 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.555969 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.556054 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.555990 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.556754 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.556723 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.557164 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.557142 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.558492 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.558466 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.558593 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.558477 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.558666 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.558600 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.558749 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.558721 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.558900 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.558857 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.558988 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.558960 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.559101 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.559080 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.559215 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.559199 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-config-volume\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.559805 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.559787 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-config-out\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.560323 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.560302 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-web-config\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.564955 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.564938 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd9wn\" (UniqueName: \"kubernetes.io/projected/cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8-kube-api-access-kd9wn\") pod \"alertmanager-main-0\" (UID: \"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.656049 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.656015 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:44:57.801584 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:57.801556 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:44:57.802730 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:44:57.802705 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcae91e4f_53e4_47cd_a9f9_0bdfa6c290e8.slice/crio-4e814fafb86539ffd992983b3ab93f42b5bd888bb097d8296c811a4c1dbb5482 WatchSource:0}: Error finding container 4e814fafb86539ffd992983b3ab93f42b5bd888bb097d8296c811a4c1dbb5482: Status 404 returned error can't find the container with id 4e814fafb86539ffd992983b3ab93f42b5bd888bb097d8296c811a4c1dbb5482 Jul 07 15:44:58.274922 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:58.274882 2575 generic.go:356] "Generic (PLEG): container finished" podID="cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8" containerID="9be1408130ade7b0b714edd121b860bfea885f217037bdd9a8b97800d6e6c709" exitCode=0 Jul 07 15:44:58.275285 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:58.274954 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8","Type":"ContainerDied","Data":"9be1408130ade7b0b714edd121b860bfea885f217037bdd9a8b97800d6e6c709"} Jul 07 15:44:58.275285 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:58.275020 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8","Type":"ContainerStarted","Data":"4e814fafb86539ffd992983b3ab93f42b5bd888bb097d8296c811a4c1dbb5482"} Jul 07 15:44:59.282357 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:59.282324 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8","Type":"ContainerStarted","Data":"d4fbe98680f2cef24e7aedbca41f6bda40aefd7870fd27fdb4f55cd933c998a0"} Jul 07 15:44:59.282787 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:59.282396 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8","Type":"ContainerStarted","Data":"d4371c2fc78dbf898d795dcee5430bad55fb471e9ec30ef9d5f2d9487de7cf72"} Jul 07 15:44:59.282787 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:59.282428 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8","Type":"ContainerStarted","Data":"1793e334261e371f080c48ee3d7ed9206bfb95da925607fa74b1c6e4ef8593af"} Jul 07 15:44:59.282787 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:59.282455 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8","Type":"ContainerStarted","Data":"70cb50ba6c78c1c0b9ac7252b5d929bb75cb7112cebbfb6deef75ccb86a65f50"} Jul 07 15:44:59.282787 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:59.282479 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8","Type":"ContainerStarted","Data":"1053a137d1b155ad9e3dea764c47f8c2ebc889bccb87c11660c73bb5f957ab69"} Jul 07 15:44:59.282787 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:59.282502 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8","Type":"ContainerStarted","Data":"29bce62901581c3481dacfb31f1a7681946d4562d4baace816efc2af18c13128"} Jul 07 15:44:59.315394 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:44:59.315307 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.291584687 podStartE2EDuration="2.315288302s" totalImagesPullingTime="0s" totalInitContainerRuntime="23.703615ms" isStatefulPod=false podCreationTimestamp="2026-07-07 15:44:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:59.315072052 +0000 UTC m=+186.410353130" watchObservedRunningTime="2026-07-07 15:44:59.315288302 +0000 UTC m=+186.410569384" Jul 07 15:45:00.136663 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:00.136625 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 15:45:00.137151 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:00.137084 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="kube-rbac-proxy" containerID="cri-o://36a1e3a5c9b676ebfe11ef63be9dd15fc2fb008c911151e6bcbf653336170abe" gracePeriod=600 Jul 07 15:45:00.137151 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:00.137099 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="thanos-sidecar" containerID="cri-o://c56af119761f6576dd4d589bf69f425b9d1b31ce2989ac8b0a33b77b7c4427f0" gracePeriod=600 Jul 07 15:45:00.137438 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:00.137123 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="kube-rbac-proxy-thanos" containerID="cri-o://5dd8e6df2be1edba1167a3e771e786ff6bb274c5eff86c4bb9e1fbdacff183e1" gracePeriod=600 Jul 07 15:45:00.137438 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:00.137086 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="config-reloader" containerID="cri-o://994e1c027f2bb272e9080cd99e14c58a6d9f9d224ac1e8f45581c298066afc3c" gracePeriod=600 Jul 07 15:45:00.137438 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:00.137110 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="kube-rbac-proxy-web" containerID="cri-o://2163add76e4cc393e705d244e7af20ef6980d02ac098eba0505062dc23ea7d8b" gracePeriod=600 Jul 07 15:45:00.137438 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:00.137333 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="prometheus" containerID="cri-o://a3d7515e0b0a774fa4220d5fa860a589e8a75e2953f17d428f71d00ce545f44a" gracePeriod=600 Jul 07 15:45:00.289320 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:00.289287 2575 generic.go:356] "Generic (PLEG): container finished" podID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerID="5dd8e6df2be1edba1167a3e771e786ff6bb274c5eff86c4bb9e1fbdacff183e1" exitCode=0 Jul 07 15:45:00.289320 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:00.289313 2575 generic.go:356] "Generic (PLEG): container finished" podID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerID="36a1e3a5c9b676ebfe11ef63be9dd15fc2fb008c911151e6bcbf653336170abe" exitCode=0 Jul 07 15:45:00.289320 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:00.289319 2575 generic.go:356] "Generic (PLEG): container finished" podID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerID="c56af119761f6576dd4d589bf69f425b9d1b31ce2989ac8b0a33b77b7c4427f0" exitCode=0 Jul 07 15:45:00.289320 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:00.289325 2575 generic.go:356] "Generic (PLEG): container finished" podID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerID="994e1c027f2bb272e9080cd99e14c58a6d9f9d224ac1e8f45581c298066afc3c" exitCode=0 Jul 07 15:45:00.289848 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:00.289332 2575 generic.go:356] "Generic (PLEG): container finished" podID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerID="a3d7515e0b0a774fa4220d5fa860a589e8a75e2953f17d428f71d00ce545f44a" exitCode=0 Jul 07 15:45:00.289848 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:00.289356 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f2e0ea8-80c9-4b02-b39d-e36e074638b6","Type":"ContainerDied","Data":"5dd8e6df2be1edba1167a3e771e786ff6bb274c5eff86c4bb9e1fbdacff183e1"} Jul 07 15:45:00.289848 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:00.289438 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f2e0ea8-80c9-4b02-b39d-e36e074638b6","Type":"ContainerDied","Data":"36a1e3a5c9b676ebfe11ef63be9dd15fc2fb008c911151e6bcbf653336170abe"} Jul 07 15:45:00.289848 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:00.289476 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f2e0ea8-80c9-4b02-b39d-e36e074638b6","Type":"ContainerDied","Data":"c56af119761f6576dd4d589bf69f425b9d1b31ce2989ac8b0a33b77b7c4427f0"} Jul 07 15:45:00.289848 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:00.289501 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f2e0ea8-80c9-4b02-b39d-e36e074638b6","Type":"ContainerDied","Data":"994e1c027f2bb272e9080cd99e14c58a6d9f9d224ac1e8f45581c298066afc3c"} Jul 07 15:45:00.289848 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:00.289529 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f2e0ea8-80c9-4b02-b39d-e36e074638b6","Type":"ContainerDied","Data":"a3d7515e0b0a774fa4220d5fa860a589e8a75e2953f17d428f71d00ce545f44a"} Jul 07 15:45:01.298595 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.298563 2575 generic.go:356] "Generic (PLEG): container finished" podID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerID="2163add76e4cc393e705d244e7af20ef6980d02ac098eba0505062dc23ea7d8b" exitCode=0 Jul 07 15:45:01.298979 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.298641 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f2e0ea8-80c9-4b02-b39d-e36e074638b6","Type":"ContainerDied","Data":"2163add76e4cc393e705d244e7af20ef6980d02ac098eba0505062dc23ea7d8b"} Jul 07 15:45:01.380703 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.380679 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:01.494913 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.494800 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-prometheus-k8s-tls\") pod \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " Jul 07 15:45:01.494913 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.494839 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-configmap-metrics-client-ca\") pod \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " Jul 07 15:45:01.494913 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.494892 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjpfw\" (UniqueName: \"kubernetes.io/projected/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-kube-api-access-wjpfw\") pod \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " Jul 07 15:45:01.494913 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.494914 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-grpc-tls\") pod \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " Jul 07 15:45:01.495231 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.494936 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-thanos-prometheus-http-client-file\") pod \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " Jul 07 15:45:01.495231 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.494964 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-metrics-client-certs\") pod \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " Jul 07 15:45:01.495231 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.494988 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-k8s-db\") pod \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " Jul 07 15:45:01.495231 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.495008 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-config-out\") pod \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " Jul 07 15:45:01.495231 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.495034 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " Jul 07 15:45:01.495231 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.495052 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-k8s-rulefiles-0\") pod \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " Jul 07 15:45:01.495231 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.495073 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-k8s-rulefiles-2\") pod \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " Jul 07 15:45:01.495231 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.495096 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-tls-assets\") pod \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " Jul 07 15:45:01.495231 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.495121 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-web-config\") pod \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " Jul 07 15:45:01.495231 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.495155 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-config\") pod \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " Jul 07 15:45:01.495231 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.495191 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-configmap-serving-certs-ca-bundle\") pod \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " Jul 07 15:45:01.495231 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.495223 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-configmap-kubelet-serving-ca-bundle\") pod \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " Jul 07 15:45:01.495908 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.495258 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-kube-rbac-proxy\") pod \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " Jul 07 15:45:01.495908 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.495292 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-trusted-ca-bundle\") pod \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " Jul 07 15:45:01.495908 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.495321 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-k8s-rulefiles-1\") pod \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " Jul 07 15:45:01.495908 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.495363 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\" (UID: \"4f2e0ea8-80c9-4b02-b39d-e36e074638b6\") " Jul 07 15:45:01.495908 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.495375 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "4f2e0ea8-80c9-4b02-b39d-e36e074638b6" (UID: "4f2e0ea8-80c9-4b02-b39d-e36e074638b6"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:45:01.495908 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.495651 2575 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-configmap-metrics-client-ca\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:45:01.497425 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.496407 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "4f2e0ea8-80c9-4b02-b39d-e36e074638b6" (UID: "4f2e0ea8-80c9-4b02-b39d-e36e074638b6"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:45:01.497425 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.497057 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "4f2e0ea8-80c9-4b02-b39d-e36e074638b6" (UID: "4f2e0ea8-80c9-4b02-b39d-e36e074638b6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:45:01.497425 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.497103 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "4f2e0ea8-80c9-4b02-b39d-e36e074638b6" (UID: "4f2e0ea8-80c9-4b02-b39d-e36e074638b6"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:45:01.497711 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.497429 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "4f2e0ea8-80c9-4b02-b39d-e36e074638b6" (UID: "4f2e0ea8-80c9-4b02-b39d-e36e074638b6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:45:01.497711 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.497600 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "4f2e0ea8-80c9-4b02-b39d-e36e074638b6" (UID: "4f2e0ea8-80c9-4b02-b39d-e36e074638b6"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:45:01.497817 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.497793 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "4f2e0ea8-80c9-4b02-b39d-e36e074638b6" (UID: "4f2e0ea8-80c9-4b02-b39d-e36e074638b6"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:45:01.498231 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.498205 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-kube-api-access-wjpfw" (OuterVolumeSpecName: "kube-api-access-wjpfw") pod "4f2e0ea8-80c9-4b02-b39d-e36e074638b6" (UID: "4f2e0ea8-80c9-4b02-b39d-e36e074638b6"). InnerVolumeSpecName "kube-api-access-wjpfw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:45:01.498323 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.498242 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-config" (OuterVolumeSpecName: "config") pod "4f2e0ea8-80c9-4b02-b39d-e36e074638b6" (UID: "4f2e0ea8-80c9-4b02-b39d-e36e074638b6"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:45:01.499105 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.499063 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "4f2e0ea8-80c9-4b02-b39d-e36e074638b6" (UID: "4f2e0ea8-80c9-4b02-b39d-e36e074638b6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:45:01.499199 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.499174 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "4f2e0ea8-80c9-4b02-b39d-e36e074638b6" (UID: "4f2e0ea8-80c9-4b02-b39d-e36e074638b6"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:45:01.499401 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.499372 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "4f2e0ea8-80c9-4b02-b39d-e36e074638b6" (UID: "4f2e0ea8-80c9-4b02-b39d-e36e074638b6"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:45:01.499486 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.499454 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-config-out" (OuterVolumeSpecName: "config-out") pod "4f2e0ea8-80c9-4b02-b39d-e36e074638b6" (UID: "4f2e0ea8-80c9-4b02-b39d-e36e074638b6"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:45:01.499600 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.499559 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "4f2e0ea8-80c9-4b02-b39d-e36e074638b6" (UID: "4f2e0ea8-80c9-4b02-b39d-e36e074638b6"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:45:01.499600 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.499579 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "4f2e0ea8-80c9-4b02-b39d-e36e074638b6" (UID: "4f2e0ea8-80c9-4b02-b39d-e36e074638b6"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:45:01.499907 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.499852 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "4f2e0ea8-80c9-4b02-b39d-e36e074638b6" (UID: "4f2e0ea8-80c9-4b02-b39d-e36e074638b6"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:45:01.500141 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.500122 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4f2e0ea8-80c9-4b02-b39d-e36e074638b6" (UID: "4f2e0ea8-80c9-4b02-b39d-e36e074638b6"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:45:01.500306 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.500282 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "4f2e0ea8-80c9-4b02-b39d-e36e074638b6" (UID: "4f2e0ea8-80c9-4b02-b39d-e36e074638b6"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:45:01.501029 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.501007 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "4f2e0ea8-80c9-4b02-b39d-e36e074638b6" (UID: "4f2e0ea8-80c9-4b02-b39d-e36e074638b6"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:45:01.510122 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.510101 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-web-config" (OuterVolumeSpecName: "web-config") pod "4f2e0ea8-80c9-4b02-b39d-e36e074638b6" (UID: "4f2e0ea8-80c9-4b02-b39d-e36e074638b6"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:45:01.596612 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.596580 2575 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-grpc-tls\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:45:01.596612 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.596608 2575 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-thanos-prometheus-http-client-file\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:45:01.596773 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.596619 2575 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-metrics-client-certs\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:45:01.596773 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.596628 2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-k8s-db\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:45:01.596773 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.596637 2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-config-out\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:45:01.596773 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.596646 2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:45:01.596773 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.596655 2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:45:01.596773 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.596663 2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:45:01.596773 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.596671 2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-tls-assets\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:45:01.596773 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.596679 2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-web-config\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:45:01.596773 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.596688 2575 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-config\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:45:01.596773 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.596695 2575 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:45:01.596773 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.596703 2575 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:45:01.596773 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.596711 2575 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-kube-rbac-proxy\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:45:01.596773 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.596720 2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-trusted-ca-bundle\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:45:01.596773 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.596728 2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:45:01.596773 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.596737 2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:45:01.596773 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.596745 2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-secret-prometheus-k8s-tls\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:45:01.596773 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:01.596753 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjpfw\" (UniqueName: \"kubernetes.io/projected/4f2e0ea8-80c9-4b02-b39d-e36e074638b6-kube-api-access-wjpfw\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 15:45:02.306435 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.306409 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.306806 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.306410 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f2e0ea8-80c9-4b02-b39d-e36e074638b6","Type":"ContainerDied","Data":"314d0b571764c249392e9a0929fd16ed10eae61cccfc2658e4e7adf1fe532f1a"} Jul 07 15:45:02.306806 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.306557 2575 scope.go:117] "RemoveContainer" containerID="5dd8e6df2be1edba1167a3e771e786ff6bb274c5eff86c4bb9e1fbdacff183e1" Jul 07 15:45:02.314208 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.314068 2575 scope.go:117] "RemoveContainer" containerID="36a1e3a5c9b676ebfe11ef63be9dd15fc2fb008c911151e6bcbf653336170abe" Jul 07 15:45:02.321318 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.321303 2575 scope.go:117] "RemoveContainer" containerID="2163add76e4cc393e705d244e7af20ef6980d02ac098eba0505062dc23ea7d8b" Jul 07 15:45:02.328222 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.328205 2575 scope.go:117] "RemoveContainer" containerID="c56af119761f6576dd4d589bf69f425b9d1b31ce2989ac8b0a33b77b7c4427f0" Jul 07 15:45:02.336554 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.336531 2575 scope.go:117] "RemoveContainer" containerID="994e1c027f2bb272e9080cd99e14c58a6d9f9d224ac1e8f45581c298066afc3c" Jul 07 15:45:02.342786 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.342764 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 15:45:02.344233 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.344215 2575 scope.go:117] "RemoveContainer" containerID="a3d7515e0b0a774fa4220d5fa860a589e8a75e2953f17d428f71d00ce545f44a" Jul 07 15:45:02.351282 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.351262 2575 scope.go:117] "RemoveContainer" containerID="4c297a31451ac07ffc7c43435e20aa8e6679aa10e841c9c9f17342dcbb81e4e1" Jul 07 15:45:02.364902 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.364859 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 15:45:02.408676 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.408644 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 15:45:02.408978 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.408966 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="init-config-reloader" Jul 07 15:45:02.409028 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.408980 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="init-config-reloader" Jul 07 15:45:02.409028 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.408997 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="kube-rbac-proxy-web" Jul 07 15:45:02.409028 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.409003 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="kube-rbac-proxy-web" Jul 07 15:45:02.409028 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.409008 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="kube-rbac-proxy" Jul 07 15:45:02.409028 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.409015 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="kube-rbac-proxy" Jul 07 15:45:02.409028 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.409019 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="kube-rbac-proxy-thanos" Jul 07 15:45:02.409028 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.409024 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="kube-rbac-proxy-thanos" Jul 07 15:45:02.409270 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.409032 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="config-reloader" Jul 07 15:45:02.409270 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.409037 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="config-reloader" Jul 07 15:45:02.409270 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.409047 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="prometheus" Jul 07 15:45:02.409270 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.409052 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="prometheus" Jul 07 15:45:02.409270 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.409062 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="thanos-sidecar" Jul 07 15:45:02.409270 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.409068 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="thanos-sidecar" Jul 07 15:45:02.409270 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.409112 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="config-reloader" Jul 07 15:45:02.409270 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.409121 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="prometheus" Jul 07 15:45:02.409270 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.409127 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="kube-rbac-proxy-web" Jul 07 15:45:02.409270 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.409132 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="thanos-sidecar" Jul 07 15:45:02.409270 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.409137 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="kube-rbac-proxy" Jul 07 15:45:02.409270 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.409143 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" containerName="kube-rbac-proxy-thanos" Jul 07 15:45:02.413654 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.413632 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.417094 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.417067 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 15:45:02.417208 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.417124 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 15:45:02.417208 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.417126 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 15:45:02.417208 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.417148 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9r4x8\"" Jul 07 15:45:02.419009 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.418990 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 15:45:02.419520 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.419507 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 15:45:02.420186 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.420169 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 15:45:02.420269 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.420223 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 15:45:02.420331 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.420233 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 15:45:02.420469 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.420445 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-amr96bvb0c8m4\"" Jul 07 15:45:02.422728 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.421830 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 15:45:02.422728 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.422246 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 15:45:02.422988 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.422920 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 15:45:02.428829 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.428806 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 15:45:02.428993 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.428882 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 15:45:02.429089 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.429076 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 15:45:02.441565 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.441543 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 15:45:02.504824 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.504731 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.504824 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.504776 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.505052 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.504836 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eed041c2-5288-42b7-a2f7-6d8cae114196-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.505052 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.504888 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/eed041c2-5288-42b7-a2f7-6d8cae114196-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.505052 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.504912 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.505052 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.504938 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.505052 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.505001 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-web-config\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.505310 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.505063 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-config\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.505310 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.505096 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.505310 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.505128 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/eed041c2-5288-42b7-a2f7-6d8cae114196-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.505310 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.505156 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.505310 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.505179 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/eed041c2-5288-42b7-a2f7-6d8cae114196-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.505310 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.505222 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eed041c2-5288-42b7-a2f7-6d8cae114196-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.505310 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.505245 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eed041c2-5288-42b7-a2f7-6d8cae114196-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.505310 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.505284 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.505310 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.505307 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lbp6\" (UniqueName: \"kubernetes.io/projected/eed041c2-5288-42b7-a2f7-6d8cae114196-kube-api-access-6lbp6\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.505607 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.505331 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eed041c2-5288-42b7-a2f7-6d8cae114196-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.505607 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.505368 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eed041c2-5288-42b7-a2f7-6d8cae114196-config-out\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.505607 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.505383 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eed041c2-5288-42b7-a2f7-6d8cae114196-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.505607 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.505403 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/eed041c2-5288-42b7-a2f7-6d8cae114196-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.606797 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.606766 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-web-config\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.606797 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.606803 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-config\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.607091 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.606825 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.607091 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.606842 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/eed041c2-5288-42b7-a2f7-6d8cae114196-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.607091 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.606875 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.607091 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.606897 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/eed041c2-5288-42b7-a2f7-6d8cae114196-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.607091 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.606946 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eed041c2-5288-42b7-a2f7-6d8cae114196-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.607091 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.606987 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eed041c2-5288-42b7-a2f7-6d8cae114196-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.607091 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.607020 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.607091 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.607043 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lbp6\" (UniqueName: \"kubernetes.io/projected/eed041c2-5288-42b7-a2f7-6d8cae114196-kube-api-access-6lbp6\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.607091 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.607067 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eed041c2-5288-42b7-a2f7-6d8cae114196-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.607559 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.607115 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eed041c2-5288-42b7-a2f7-6d8cae114196-config-out\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.607559 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.607139 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eed041c2-5288-42b7-a2f7-6d8cae114196-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.607559 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.607174 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/eed041c2-5288-42b7-a2f7-6d8cae114196-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.607559 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.607202 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.607559 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.607235 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.607559 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.607269 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eed041c2-5288-42b7-a2f7-6d8cae114196-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.607559 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.607296 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/eed041c2-5288-42b7-a2f7-6d8cae114196-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.607559 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.607338 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.607559 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.607379 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.608017 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.607581 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/eed041c2-5288-42b7-a2f7-6d8cae114196-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.608017 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.607716 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eed041c2-5288-42b7-a2f7-6d8cae114196-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.608565 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.608300 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eed041c2-5288-42b7-a2f7-6d8cae114196-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.608729 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.608603 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/eed041c2-5288-42b7-a2f7-6d8cae114196-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.610632 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.608931 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eed041c2-5288-42b7-a2f7-6d8cae114196-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.610632 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.609761 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/eed041c2-5288-42b7-a2f7-6d8cae114196-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.610632 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.609886 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eed041c2-5288-42b7-a2f7-6d8cae114196-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.610632 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.610198 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.610632 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.610302 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-config\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.610632 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.610596 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-web-config\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.611368 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.611313 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.611890 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.611845 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/eed041c2-5288-42b7-a2f7-6d8cae114196-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.611992 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.611921 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.612057 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.612029 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eed041c2-5288-42b7-a2f7-6d8cae114196-config-out\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.612248 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.612230 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.612330 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.612311 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.612734 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.612706 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.612734 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.612729 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eed041c2-5288-42b7-a2f7-6d8cae114196-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.613647 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.613626 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/eed041c2-5288-42b7-a2f7-6d8cae114196-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.617725 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.617690 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lbp6\" (UniqueName: \"kubernetes.io/projected/eed041c2-5288-42b7-a2f7-6d8cae114196-kube-api-access-6lbp6\") pod \"prometheus-k8s-0\" (UID: \"eed041c2-5288-42b7-a2f7-6d8cae114196\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.724618 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.724587 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:45:02.866749 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:02.866721 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 15:45:02.869326 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:45:02.869300 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeed041c2_5288_42b7_a2f7_6d8cae114196.slice/crio-cd9af04afdf34d51d93d6af1ce7a2516ace0d26254c7abc9f777fdb754cf30ef WatchSource:0}: Error finding container cd9af04afdf34d51d93d6af1ce7a2516ace0d26254c7abc9f777fdb754cf30ef: Status 404 returned error can't find the container with id cd9af04afdf34d51d93d6af1ce7a2516ace0d26254c7abc9f777fdb754cf30ef Jul 07 15:45:03.311927 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:03.311887 2575 generic.go:356] "Generic (PLEG): container finished" podID="eed041c2-5288-42b7-a2f7-6d8cae114196" containerID="0d015425046c377f80d912c0242a6428161159290da536f90cf22ae1c708ddc9" exitCode=0 Jul 07 15:45:03.312300 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:03.311973 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eed041c2-5288-42b7-a2f7-6d8cae114196","Type":"ContainerDied","Data":"0d015425046c377f80d912c0242a6428161159290da536f90cf22ae1c708ddc9"} Jul 07 15:45:03.312300 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:03.312052 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eed041c2-5288-42b7-a2f7-6d8cae114196","Type":"ContainerStarted","Data":"cd9af04afdf34d51d93d6af1ce7a2516ace0d26254c7abc9f777fdb754cf30ef"} Jul 07 15:45:03.435813 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:03.435781 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f2e0ea8-80c9-4b02-b39d-e36e074638b6" path="/var/lib/kubelet/pods/4f2e0ea8-80c9-4b02-b39d-e36e074638b6/volumes" Jul 07 15:45:04.320065 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:04.320027 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eed041c2-5288-42b7-a2f7-6d8cae114196","Type":"ContainerStarted","Data":"e83ea312c4f2d6405c5df82bdfd9e95f28c4a482b76a84c8c52e96681d3862a0"} Jul 07 15:45:04.320451 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:04.320118 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eed041c2-5288-42b7-a2f7-6d8cae114196","Type":"ContainerStarted","Data":"77bef560d311df021ab4f47938381391a345d19c6cd1a8216024d2bc0edc3443"} Jul 07 15:45:04.320451 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:04.320145 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eed041c2-5288-42b7-a2f7-6d8cae114196","Type":"ContainerStarted","Data":"80b85abedd6aae81b3398c8de1ea4cd2e7e82e413f59b51f087a6183a4220e12"} Jul 07 15:45:04.320451 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:04.320169 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eed041c2-5288-42b7-a2f7-6d8cae114196","Type":"ContainerStarted","Data":"0470d8c9846ebad5322b70ed104201c35d5ad7cdc0f73b3fde3905cbfbefe978"} Jul 07 15:45:04.320451 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:04.320192 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eed041c2-5288-42b7-a2f7-6d8cae114196","Type":"ContainerStarted","Data":"e297de22607554e9f95bb4f979ff7efa7becb89f744606953cad2abeb6e42080"} Jul 07 15:45:04.320451 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:04.320215 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eed041c2-5288-42b7-a2f7-6d8cae114196","Type":"ContainerStarted","Data":"c3c303f0b411e2f2933363ca1213eb06c597596294a373fda038bdcae1c1d791"} Jul 07 15:45:04.355130 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:04.355078 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.336927992 podStartE2EDuration="2.355062544s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.134552ms" isStatefulPod=false podCreationTimestamp="2026-07-07 15:45:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:04.353090427 +0000 UTC m=+191.448371510" watchObservedRunningTime="2026-07-07 15:45:04.355062544 +0000 UTC m=+191.450343659" Jul 07 15:45:07.725271 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:45:07.725231 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:02.725674 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:02.725628 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:02.741307 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:02.741281 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:03.565406 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:03.565378 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:46:47.469795 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:47.469761 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-p969b"] Jul 07 15:46:47.472939 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:47.472922 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-p969b" Jul 07 15:46:47.476312 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:47.476288 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 15:46:47.476442 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:47.476288 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 15:46:47.477399 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:47.477380 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-q4pks\"" Jul 07 15:46:47.485469 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:47.485450 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-p969b"] Jul 07 15:46:47.574296 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:47.574268 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czhwk\" (UniqueName: \"kubernetes.io/projected/cfdc1303-eb80-426d-92b3-1935ad2f8eb6-kube-api-access-czhwk\") pod \"cert-manager-55b4d954d5-p969b\" (UID: \"cfdc1303-eb80-426d-92b3-1935ad2f8eb6\") " pod="cert-manager/cert-manager-55b4d954d5-p969b" Jul 07 15:46:47.574465 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:47.574372 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cfdc1303-eb80-426d-92b3-1935ad2f8eb6-bound-sa-token\") pod \"cert-manager-55b4d954d5-p969b\" (UID: \"cfdc1303-eb80-426d-92b3-1935ad2f8eb6\") " pod="cert-manager/cert-manager-55b4d954d5-p969b" Jul 07 15:46:47.675238 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:47.675206 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cfdc1303-eb80-426d-92b3-1935ad2f8eb6-bound-sa-token\") pod \"cert-manager-55b4d954d5-p969b\" (UID: \"cfdc1303-eb80-426d-92b3-1935ad2f8eb6\") " pod="cert-manager/cert-manager-55b4d954d5-p969b" Jul 07 15:46:47.675337 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:47.675258 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czhwk\" (UniqueName: \"kubernetes.io/projected/cfdc1303-eb80-426d-92b3-1935ad2f8eb6-kube-api-access-czhwk\") pod \"cert-manager-55b4d954d5-p969b\" (UID: \"cfdc1303-eb80-426d-92b3-1935ad2f8eb6\") " pod="cert-manager/cert-manager-55b4d954d5-p969b" Jul 07 15:46:47.685733 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:47.685707 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cfdc1303-eb80-426d-92b3-1935ad2f8eb6-bound-sa-token\") pod \"cert-manager-55b4d954d5-p969b\" (UID: \"cfdc1303-eb80-426d-92b3-1935ad2f8eb6\") " pod="cert-manager/cert-manager-55b4d954d5-p969b" Jul 07 15:46:47.685886 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:47.685795 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czhwk\" (UniqueName: \"kubernetes.io/projected/cfdc1303-eb80-426d-92b3-1935ad2f8eb6-kube-api-access-czhwk\") pod \"cert-manager-55b4d954d5-p969b\" (UID: \"cfdc1303-eb80-426d-92b3-1935ad2f8eb6\") " pod="cert-manager/cert-manager-55b4d954d5-p969b" Jul 07 15:46:47.782050 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:47.781966 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-p969b" Jul 07 15:46:47.912183 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:47.912160 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-p969b"] Jul 07 15:46:47.914513 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:46:47.914484 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcfdc1303_eb80_426d_92b3_1935ad2f8eb6.slice/crio-c85e4474f0c141d10b5d9e0f4109e96e1aa4642864b4c1a64177c13e14886c23 WatchSource:0}: Error finding container c85e4474f0c141d10b5d9e0f4109e96e1aa4642864b4c1a64177c13e14886c23: Status 404 returned error can't find the container with id c85e4474f0c141d10b5d9e0f4109e96e1aa4642864b4c1a64177c13e14886c23 Jul 07 15:46:48.735028 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:48.734989 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-p969b" event={"ID":"cfdc1303-eb80-426d-92b3-1935ad2f8eb6","Type":"ContainerStarted","Data":"c85e4474f0c141d10b5d9e0f4109e96e1aa4642864b4c1a64177c13e14886c23"} Jul 07 15:46:51.751531 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:51.751491 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-p969b" event={"ID":"cfdc1303-eb80-426d-92b3-1935ad2f8eb6","Type":"ContainerStarted","Data":"c1bf27ae0ee8061626a288e62c6b8f8aeb58d537ac957e81a470ce6b88b869aa"} Jul 07 15:46:51.770577 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:51.770525 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-p969b" podStartSLOduration=1.388607335 podStartE2EDuration="4.770511114s" totalImagesPullingTime="3.381903779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:46:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:46:51.769216161 +0000 UTC m=+298.864497240" watchObservedRunningTime="2026-07-07 15:46:51.770511114 +0000 UTC m=+298.865792194" Jul 07 15:46:53.310558 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:53.310526 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nfh8b_162f95df-a98f-49a6-a91e-689a8e22be46/console-operator/1.log" Jul 07 15:46:53.316777 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:53.316751 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nfh8b_162f95df-a98f-49a6-a91e-689a8e22be46/console-operator/1.log" Jul 07 15:46:53.317553 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:53.317529 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4d86s_fbf78c29-89f3-4aad-82a7-2e8f99b6356e/ovn-acl-logging/0.log" Jul 07 15:46:53.321240 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:53.321221 2575 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 15:46:53.330084 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:53.330061 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4d86s_fbf78c29-89f3-4aad-82a7-2e8f99b6356e/ovn-acl-logging/0.log" Jul 07 15:46:55.030297 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:55.030260 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-86b7f856ff-jbr6g"] Jul 07 15:46:55.033738 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:55.033712 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-86b7f856ff-jbr6g" Jul 07 15:46:55.037966 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:55.037935 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 15:46:55.037966 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:55.037946 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 07 15:46:55.041717 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:55.041698 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:46:55.042562 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:55.042545 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ppjfm\"" Jul 07 15:46:55.042659 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:55.042575 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 07 15:46:55.042659 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:55.042549 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 07 15:46:55.062626 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:55.062596 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-86b7f856ff-jbr6g"] Jul 07 15:46:55.144105 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:55.144070 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1e471450-c3bf-4332-9fda-fd14594f7150-manager-config\") pod \"lws-controller-manager-86b7f856ff-jbr6g\" (UID: \"1e471450-c3bf-4332-9fda-fd14594f7150\") " pod="openshift-lws-operator/lws-controller-manager-86b7f856ff-jbr6g" Jul 07 15:46:55.144248 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:55.144109 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfgj9\" (UniqueName: \"kubernetes.io/projected/1e471450-c3bf-4332-9fda-fd14594f7150-kube-api-access-xfgj9\") pod \"lws-controller-manager-86b7f856ff-jbr6g\" (UID: \"1e471450-c3bf-4332-9fda-fd14594f7150\") " pod="openshift-lws-operator/lws-controller-manager-86b7f856ff-jbr6g" Jul 07 15:46:55.144248 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:55.144132 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e471450-c3bf-4332-9fda-fd14594f7150-cert\") pod \"lws-controller-manager-86b7f856ff-jbr6g\" (UID: \"1e471450-c3bf-4332-9fda-fd14594f7150\") " pod="openshift-lws-operator/lws-controller-manager-86b7f856ff-jbr6g" Jul 07 15:46:55.144248 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:55.144148 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1e471450-c3bf-4332-9fda-fd14594f7150-metrics-cert\") pod \"lws-controller-manager-86b7f856ff-jbr6g\" (UID: \"1e471450-c3bf-4332-9fda-fd14594f7150\") " pod="openshift-lws-operator/lws-controller-manager-86b7f856ff-jbr6g" Jul 07 15:46:55.245381 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:55.245347 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1e471450-c3bf-4332-9fda-fd14594f7150-manager-config\") pod \"lws-controller-manager-86b7f856ff-jbr6g\" (UID: \"1e471450-c3bf-4332-9fda-fd14594f7150\") " pod="openshift-lws-operator/lws-controller-manager-86b7f856ff-jbr6g" Jul 07 15:46:55.245381 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:55.245382 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfgj9\" (UniqueName: \"kubernetes.io/projected/1e471450-c3bf-4332-9fda-fd14594f7150-kube-api-access-xfgj9\") pod \"lws-controller-manager-86b7f856ff-jbr6g\" (UID: \"1e471450-c3bf-4332-9fda-fd14594f7150\") " pod="openshift-lws-operator/lws-controller-manager-86b7f856ff-jbr6g" Jul 07 15:46:55.245568 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:55.245402 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e471450-c3bf-4332-9fda-fd14594f7150-cert\") pod \"lws-controller-manager-86b7f856ff-jbr6g\" (UID: \"1e471450-c3bf-4332-9fda-fd14594f7150\") " pod="openshift-lws-operator/lws-controller-manager-86b7f856ff-jbr6g" Jul 07 15:46:55.245568 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:55.245416 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1e471450-c3bf-4332-9fda-fd14594f7150-metrics-cert\") pod \"lws-controller-manager-86b7f856ff-jbr6g\" (UID: \"1e471450-c3bf-4332-9fda-fd14594f7150\") " pod="openshift-lws-operator/lws-controller-manager-86b7f856ff-jbr6g" Jul 07 15:46:55.246022 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:55.246000 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1e471450-c3bf-4332-9fda-fd14594f7150-manager-config\") pod \"lws-controller-manager-86b7f856ff-jbr6g\" (UID: \"1e471450-c3bf-4332-9fda-fd14594f7150\") " pod="openshift-lws-operator/lws-controller-manager-86b7f856ff-jbr6g" Jul 07 15:46:55.247937 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:55.247916 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e471450-c3bf-4332-9fda-fd14594f7150-cert\") pod \"lws-controller-manager-86b7f856ff-jbr6g\" (UID: \"1e471450-c3bf-4332-9fda-fd14594f7150\") " pod="openshift-lws-operator/lws-controller-manager-86b7f856ff-jbr6g" Jul 07 15:46:55.248021 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:55.247957 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1e471450-c3bf-4332-9fda-fd14594f7150-metrics-cert\") pod \"lws-controller-manager-86b7f856ff-jbr6g\" (UID: \"1e471450-c3bf-4332-9fda-fd14594f7150\") " pod="openshift-lws-operator/lws-controller-manager-86b7f856ff-jbr6g" Jul 07 15:46:55.254410 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:55.254390 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfgj9\" (UniqueName: \"kubernetes.io/projected/1e471450-c3bf-4332-9fda-fd14594f7150-kube-api-access-xfgj9\") pod \"lws-controller-manager-86b7f856ff-jbr6g\" (UID: \"1e471450-c3bf-4332-9fda-fd14594f7150\") " pod="openshift-lws-operator/lws-controller-manager-86b7f856ff-jbr6g" Jul 07 15:46:55.343259 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:55.343235 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-86b7f856ff-jbr6g" Jul 07 15:46:55.477816 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:55.477791 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-86b7f856ff-jbr6g"] Jul 07 15:46:55.479652 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:46:55.479622 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e471450_c3bf_4332_9fda_fd14594f7150.slice/crio-7c6bad1dae298be040de3d553d8186ad0760bfb75175947b5c519c4b4f43a8ea WatchSource:0}: Error finding container 7c6bad1dae298be040de3d553d8186ad0760bfb75175947b5c519c4b4f43a8ea: Status 404 returned error can't find the container with id 7c6bad1dae298be040de3d553d8186ad0760bfb75175947b5c519c4b4f43a8ea Jul 07 15:46:55.481616 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:55.481599 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 15:46:55.768452 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:55.768372 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-86b7f856ff-jbr6g" event={"ID":"1e471450-c3bf-4332-9fda-fd14594f7150","Type":"ContainerStarted","Data":"7c6bad1dae298be040de3d553d8186ad0760bfb75175947b5c519c4b4f43a8ea"} Jul 07 15:46:58.781676 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:58.781636 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-86b7f856ff-jbr6g" event={"ID":"1e471450-c3bf-4332-9fda-fd14594f7150","Type":"ContainerStarted","Data":"1e8f8b0e6d3daa24acf47660d997fb226832a8d0ad1621ce14ed1859a6180e31"} Jul 07 15:46:58.782114 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:58.781775 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-86b7f856ff-jbr6g" Jul 07 15:46:58.804856 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:46:58.804806 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-86b7f856ff-jbr6g" podStartSLOduration=1.458307972 podStartE2EDuration="3.804789529s" totalImagesPullingTime="2.346481557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:46:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:46:58.803282581 +0000 UTC m=+305.898563660" watchObservedRunningTime="2026-07-07 15:46:58.804789529 +0000 UTC m=+305.900070607" Jul 07 15:47:00.648927 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:00.648894 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r"] Jul 07 15:47:00.652521 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:00.652504 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r" Jul 07 15:47:00.656319 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:00.656296 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 15:47:00.656524 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:00.656336 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 15:47:00.656630 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:00.656587 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 15:47:00.656701 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:00.656673 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hvlrf\"" Jul 07 15:47:00.656906 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:00.656890 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 15:47:00.677696 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:00.677653 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r"] Jul 07 15:47:00.798502 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:00.798468 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1dcf30c8-4b0b-4da9-9aa4-99865a731b52-manifests\") pod \"opendatahub-operator-controller-manager-865bb77d7b-mcj5r\" (UID: \"1dcf30c8-4b0b-4da9-9aa4-99865a731b52\") " pod="opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r" Jul 07 15:47:00.798502 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:00.798503 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjh6z\" (UniqueName: \"kubernetes.io/projected/1dcf30c8-4b0b-4da9-9aa4-99865a731b52-kube-api-access-vjh6z\") pod \"opendatahub-operator-controller-manager-865bb77d7b-mcj5r\" (UID: \"1dcf30c8-4b0b-4da9-9aa4-99865a731b52\") " pod="opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r" Jul 07 15:47:00.798726 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:00.798596 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1dcf30c8-4b0b-4da9-9aa4-99865a731b52-tmp\") pod \"opendatahub-operator-controller-manager-865bb77d7b-mcj5r\" (UID: \"1dcf30c8-4b0b-4da9-9aa4-99865a731b52\") " pod="opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r" Jul 07 15:47:00.798726 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:00.798616 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1dcf30c8-4b0b-4da9-9aa4-99865a731b52-apiservice-cert\") pod \"opendatahub-operator-controller-manager-865bb77d7b-mcj5r\" (UID: \"1dcf30c8-4b0b-4da9-9aa4-99865a731b52\") " pod="opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r" Jul 07 15:47:00.798726 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:00.798642 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1dcf30c8-4b0b-4da9-9aa4-99865a731b52-webhook-cert\") pod \"opendatahub-operator-controller-manager-865bb77d7b-mcj5r\" (UID: \"1dcf30c8-4b0b-4da9-9aa4-99865a731b52\") " pod="opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r" Jul 07 15:47:00.899429 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:00.899337 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1dcf30c8-4b0b-4da9-9aa4-99865a731b52-tmp\") pod \"opendatahub-operator-controller-manager-865bb77d7b-mcj5r\" (UID: \"1dcf30c8-4b0b-4da9-9aa4-99865a731b52\") " pod="opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r" Jul 07 15:47:00.899429 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:00.899378 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1dcf30c8-4b0b-4da9-9aa4-99865a731b52-apiservice-cert\") pod \"opendatahub-operator-controller-manager-865bb77d7b-mcj5r\" (UID: \"1dcf30c8-4b0b-4da9-9aa4-99865a731b52\") " pod="opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r" Jul 07 15:47:00.899429 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:00.899405 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1dcf30c8-4b0b-4da9-9aa4-99865a731b52-webhook-cert\") pod \"opendatahub-operator-controller-manager-865bb77d7b-mcj5r\" (UID: \"1dcf30c8-4b0b-4da9-9aa4-99865a731b52\") " pod="opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r" Jul 07 15:47:00.899691 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:00.899495 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1dcf30c8-4b0b-4da9-9aa4-99865a731b52-manifests\") pod \"opendatahub-operator-controller-manager-865bb77d7b-mcj5r\" (UID: \"1dcf30c8-4b0b-4da9-9aa4-99865a731b52\") " pod="opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r" Jul 07 15:47:00.899691 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:00.899525 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjh6z\" (UniqueName: \"kubernetes.io/projected/1dcf30c8-4b0b-4da9-9aa4-99865a731b52-kube-api-access-vjh6z\") pod \"opendatahub-operator-controller-manager-865bb77d7b-mcj5r\" (UID: \"1dcf30c8-4b0b-4da9-9aa4-99865a731b52\") " pod="opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r" Jul 07 15:47:00.899953 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:00.899930 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1dcf30c8-4b0b-4da9-9aa4-99865a731b52-manifests\") pod \"opendatahub-operator-controller-manager-865bb77d7b-mcj5r\" (UID: \"1dcf30c8-4b0b-4da9-9aa4-99865a731b52\") " pod="opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r" Jul 07 15:47:00.901790 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:00.901757 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1dcf30c8-4b0b-4da9-9aa4-99865a731b52-tmp\") pod \"opendatahub-operator-controller-manager-865bb77d7b-mcj5r\" (UID: \"1dcf30c8-4b0b-4da9-9aa4-99865a731b52\") " pod="opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r" Jul 07 15:47:00.901930 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:00.901908 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1dcf30c8-4b0b-4da9-9aa4-99865a731b52-apiservice-cert\") pod \"opendatahub-operator-controller-manager-865bb77d7b-mcj5r\" (UID: \"1dcf30c8-4b0b-4da9-9aa4-99865a731b52\") " pod="opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r" Jul 07 15:47:00.901994 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:00.901922 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1dcf30c8-4b0b-4da9-9aa4-99865a731b52-webhook-cert\") pod \"opendatahub-operator-controller-manager-865bb77d7b-mcj5r\" (UID: \"1dcf30c8-4b0b-4da9-9aa4-99865a731b52\") " pod="opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r" Jul 07 15:47:00.911318 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:00.911289 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjh6z\" (UniqueName: \"kubernetes.io/projected/1dcf30c8-4b0b-4da9-9aa4-99865a731b52-kube-api-access-vjh6z\") pod \"opendatahub-operator-controller-manager-865bb77d7b-mcj5r\" (UID: \"1dcf30c8-4b0b-4da9-9aa4-99865a731b52\") " pod="opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r" Jul 07 15:47:00.963453 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:00.963418 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r" Jul 07 15:47:01.099165 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:01.099142 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r"] Jul 07 15:47:01.101084 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:47:01.101057 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dcf30c8_4b0b_4da9_9aa4_99865a731b52.slice/crio-9ea913f0b90d7d592359d553089be6c7f3d18e6119cf941efab7f6460be068d4 WatchSource:0}: Error finding container 9ea913f0b90d7d592359d553089be6c7f3d18e6119cf941efab7f6460be068d4: Status 404 returned error can't find the container with id 9ea913f0b90d7d592359d553089be6c7f3d18e6119cf941efab7f6460be068d4 Jul 07 15:47:01.796581 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:01.796541 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r" event={"ID":"1dcf30c8-4b0b-4da9-9aa4-99865a731b52","Type":"ContainerStarted","Data":"9ea913f0b90d7d592359d553089be6c7f3d18e6119cf941efab7f6460be068d4"} Jul 07 15:47:04.813960 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:04.813924 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r" event={"ID":"1dcf30c8-4b0b-4da9-9aa4-99865a731b52","Type":"ContainerStarted","Data":"f51df68b8f5d303c8e85fd9204f6d5c080eb589f53226cb12fb60cd21a4e55e6"} Jul 07 15:47:05.820179 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:05.820074 2575 generic.go:356] "Generic (PLEG): container finished" podID="1dcf30c8-4b0b-4da9-9aa4-99865a731b52" containerID="f51df68b8f5d303c8e85fd9204f6d5c080eb589f53226cb12fb60cd21a4e55e6" exitCode=0 Jul 07 15:47:05.820179 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:05.820121 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r" event={"ID":"1dcf30c8-4b0b-4da9-9aa4-99865a731b52","Type":"ContainerDied","Data":"f51df68b8f5d303c8e85fd9204f6d5c080eb589f53226cb12fb60cd21a4e55e6"} Jul 07 15:47:06.826028 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:06.825993 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r" event={"ID":"1dcf30c8-4b0b-4da9-9aa4-99865a731b52","Type":"ContainerStarted","Data":"7813064d2b8d25b7bf23706dfb531739b3545e44530d48d290b254360a05094f"} Jul 07 15:47:06.826406 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:06.826117 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r" Jul 07 15:47:06.851982 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:06.851933 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r" podStartSLOduration=2.197050638 podStartE2EDuration="6.851918853s" totalImagesPullingTime="3.627691683s" totalInitContainerRuntime="1.027176532s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:47:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:47:06.850042004 +0000 UTC m=+313.945323074" watchObservedRunningTime="2026-07-07 15:47:06.851918853 +0000 UTC m=+313.947199932" Jul 07 15:47:09.788124 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:09.788089 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-86b7f856ff-jbr6g" Jul 07 15:47:17.833144 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:17.833112 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-865bb77d7b-mcj5r" Jul 07 15:47:32.257380 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.257346 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx"] Jul 07 15:47:32.267035 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.267017 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" Jul 07 15:47:32.271390 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.271363 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 07 15:47:32.271509 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.271397 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 07 15:47:32.271509 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.271412 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 15:47:32.271509 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.271402 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 15:47:32.271761 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.271745 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 15:47:32.272118 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.272095 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-w9sbn\"" Jul 07 15:47:32.272367 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.272352 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 07 15:47:32.290287 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.290267 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx"] Jul 07 15:47:32.389231 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.389195 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc452\" (UniqueName: \"kubernetes.io/projected/c097cd64-a2ac-4a57-a49b-83c559abd38d-kube-api-access-rc452\") pod \"istiod-openshift-gateway-94bb8fbfd-tltvx\" (UID: \"c097cd64-a2ac-4a57-a49b-83c559abd38d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" Jul 07 15:47:32.389231 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.389234 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c097cd64-a2ac-4a57-a49b-83c559abd38d-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-tltvx\" (UID: \"c097cd64-a2ac-4a57-a49b-83c559abd38d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" Jul 07 15:47:32.389430 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.389268 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c097cd64-a2ac-4a57-a49b-83c559abd38d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-tltvx\" (UID: \"c097cd64-a2ac-4a57-a49b-83c559abd38d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" Jul 07 15:47:32.389430 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.389307 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c097cd64-a2ac-4a57-a49b-83c559abd38d-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-tltvx\" (UID: \"c097cd64-a2ac-4a57-a49b-83c559abd38d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" Jul 07 15:47:32.389430 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.389383 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c097cd64-a2ac-4a57-a49b-83c559abd38d-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-tltvx\" (UID: \"c097cd64-a2ac-4a57-a49b-83c559abd38d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" Jul 07 15:47:32.389430 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.389411 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c097cd64-a2ac-4a57-a49b-83c559abd38d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-tltvx\" (UID: \"c097cd64-a2ac-4a57-a49b-83c559abd38d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" Jul 07 15:47:32.389579 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.389450 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c097cd64-a2ac-4a57-a49b-83c559abd38d-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-tltvx\" (UID: \"c097cd64-a2ac-4a57-a49b-83c559abd38d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" Jul 07 15:47:32.490766 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.490730 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc452\" (UniqueName: \"kubernetes.io/projected/c097cd64-a2ac-4a57-a49b-83c559abd38d-kube-api-access-rc452\") pod \"istiod-openshift-gateway-94bb8fbfd-tltvx\" (UID: \"c097cd64-a2ac-4a57-a49b-83c559abd38d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" Jul 07 15:47:32.490942 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.490792 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c097cd64-a2ac-4a57-a49b-83c559abd38d-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-tltvx\" (UID: \"c097cd64-a2ac-4a57-a49b-83c559abd38d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" Jul 07 15:47:32.490942 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.490908 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c097cd64-a2ac-4a57-a49b-83c559abd38d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-tltvx\" (UID: \"c097cd64-a2ac-4a57-a49b-83c559abd38d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" Jul 07 15:47:32.490942 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.490938 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c097cd64-a2ac-4a57-a49b-83c559abd38d-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-tltvx\" (UID: \"c097cd64-a2ac-4a57-a49b-83c559abd38d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" Jul 07 15:47:32.491131 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.490976 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c097cd64-a2ac-4a57-a49b-83c559abd38d-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-tltvx\" (UID: \"c097cd64-a2ac-4a57-a49b-83c559abd38d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" Jul 07 15:47:32.491131 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.491003 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c097cd64-a2ac-4a57-a49b-83c559abd38d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-tltvx\" (UID: \"c097cd64-a2ac-4a57-a49b-83c559abd38d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" Jul 07 15:47:32.491131 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.491113 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c097cd64-a2ac-4a57-a49b-83c559abd38d-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-tltvx\" (UID: \"c097cd64-a2ac-4a57-a49b-83c559abd38d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" Jul 07 15:47:32.492357 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.492331 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c097cd64-a2ac-4a57-a49b-83c559abd38d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-tltvx\" (UID: \"c097cd64-a2ac-4a57-a49b-83c559abd38d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" Jul 07 15:47:32.493859 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.493833 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c097cd64-a2ac-4a57-a49b-83c559abd38d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-tltvx\" (UID: \"c097cd64-a2ac-4a57-a49b-83c559abd38d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" Jul 07 15:47:32.493967 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.493917 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c097cd64-a2ac-4a57-a49b-83c559abd38d-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-tltvx\" (UID: \"c097cd64-a2ac-4a57-a49b-83c559abd38d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" Jul 07 15:47:32.494007 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.493985 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c097cd64-a2ac-4a57-a49b-83c559abd38d-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-tltvx\" (UID: \"c097cd64-a2ac-4a57-a49b-83c559abd38d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" Jul 07 15:47:32.494191 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.494171 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c097cd64-a2ac-4a57-a49b-83c559abd38d-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-tltvx\" (UID: \"c097cd64-a2ac-4a57-a49b-83c559abd38d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" Jul 07 15:47:32.506284 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.506258 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c097cd64-a2ac-4a57-a49b-83c559abd38d-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-tltvx\" (UID: \"c097cd64-a2ac-4a57-a49b-83c559abd38d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" Jul 07 15:47:32.507841 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.507783 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc452\" (UniqueName: \"kubernetes.io/projected/c097cd64-a2ac-4a57-a49b-83c559abd38d-kube-api-access-rc452\") pod \"istiod-openshift-gateway-94bb8fbfd-tltvx\" (UID: \"c097cd64-a2ac-4a57-a49b-83c559abd38d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" Jul 07 15:47:32.576552 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.576524 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" Jul 07 15:47:32.769696 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.769605 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx"] Jul 07 15:47:32.773122 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:47:32.773095 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc097cd64_a2ac_4a57_a49b_83c559abd38d.slice/crio-7d76e01f20aa2a776831b189bf573ce92676bbf63b65b23622286aee81ad8183 WatchSource:0}: Error finding container 7d76e01f20aa2a776831b189bf573ce92676bbf63b65b23622286aee81ad8183: Status 404 returned error can't find the container with id 7d76e01f20aa2a776831b189bf573ce92676bbf63b65b23622286aee81ad8183 Jul 07 15:47:32.942487 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:32.942452 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" event={"ID":"c097cd64-a2ac-4a57-a49b-83c559abd38d","Type":"ContainerStarted","Data":"7d76e01f20aa2a776831b189bf573ce92676bbf63b65b23622286aee81ad8183"} Jul 07 15:47:34.958780 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:34.958738 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:47:34.959097 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:34.958818 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:47:35.958520 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:35.958484 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" event={"ID":"c097cd64-a2ac-4a57-a49b-83c559abd38d","Type":"ContainerStarted","Data":"58b1fdfe713881002a6e2ba904c958d49567e6f4736d56e61dd6cfbf7c7e5be6"} Jul 07 15:47:35.958778 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:35.958624 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" Jul 07 15:47:35.997379 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:35.997331 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" podStartSLOduration=1.8139401670000002 podStartE2EDuration="3.997316967s" totalImagesPullingTime="2.1833768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:47:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:47:35.996125074 +0000 UTC m=+343.091406153" watchObservedRunningTime="2026-07-07 15:47:35.997316967 +0000 UTC m=+343.092598047" Jul 07 15:47:36.964417 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:47:36.964390 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tltvx" Jul 07 15:49:05.236265 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:49:05.236228 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-tcvfn"] Jul 07 15:49:05.239562 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:49:05.239546 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tcvfn" Jul 07 15:49:05.242530 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:49:05.242507 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 15:49:05.242671 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:49:05.242540 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 15:49:05.242671 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:49:05.242666 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-fm2z7\"" Jul 07 15:49:05.258838 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:49:05.258816 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-tcvfn"] Jul 07 15:49:05.295286 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:49:05.295259 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnhsf\" (UniqueName: \"kubernetes.io/projected/55f8e5a0-a309-4faa-89bc-b3748e880df2-kube-api-access-fnhsf\") pod \"authorino-operator-75cc8d5cb7-tcvfn\" (UID: \"55f8e5a0-a309-4faa-89bc-b3748e880df2\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-tcvfn" Jul 07 15:49:05.395956 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:49:05.395922 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnhsf\" (UniqueName: \"kubernetes.io/projected/55f8e5a0-a309-4faa-89bc-b3748e880df2-kube-api-access-fnhsf\") pod \"authorino-operator-75cc8d5cb7-tcvfn\" (UID: \"55f8e5a0-a309-4faa-89bc-b3748e880df2\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-tcvfn" Jul 07 15:49:05.408620 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:49:05.408592 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnhsf\" (UniqueName: \"kubernetes.io/projected/55f8e5a0-a309-4faa-89bc-b3748e880df2-kube-api-access-fnhsf\") pod \"authorino-operator-75cc8d5cb7-tcvfn\" (UID: \"55f8e5a0-a309-4faa-89bc-b3748e880df2\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-tcvfn" Jul 07 15:49:05.549895 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:49:05.549799 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tcvfn" Jul 07 15:49:05.677757 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:49:05.677731 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-tcvfn"] Jul 07 15:49:05.681001 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:49:05.680963 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55f8e5a0_a309_4faa_89bc_b3748e880df2.slice/crio-8adbf22bf60774621866dbd59c4de31ef7b0b4837904ec0efa530c3d1eef7f9f WatchSource:0}: Error finding container 8adbf22bf60774621866dbd59c4de31ef7b0b4837904ec0efa530c3d1eef7f9f: Status 404 returned error can't find the container with id 8adbf22bf60774621866dbd59c4de31ef7b0b4837904ec0efa530c3d1eef7f9f Jul 07 15:49:06.352947 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:49:06.352894 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tcvfn" event={"ID":"55f8e5a0-a309-4faa-89bc-b3748e880df2","Type":"ContainerStarted","Data":"8adbf22bf60774621866dbd59c4de31ef7b0b4837904ec0efa530c3d1eef7f9f"} Jul 07 15:49:08.363581 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:49:08.363547 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tcvfn" event={"ID":"55f8e5a0-a309-4faa-89bc-b3748e880df2","Type":"ContainerStarted","Data":"319a39d3745f470bca3e6b1c0e34555889124f619c025d21a81157c1189ff96a"} Jul 07 15:49:08.364019 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:49:08.363690 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tcvfn" Jul 07 15:49:08.388222 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:49:08.388174 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tcvfn" podStartSLOduration=1.307210748 podStartE2EDuration="3.388159115s" totalImagesPullingTime="2.080948367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:49:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:49:08.386761219 +0000 UTC m=+435.482042298" watchObservedRunningTime="2026-07-07 15:49:08.388159115 +0000 UTC m=+435.483440193" Jul 07 15:49:19.370788 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:49:19.370707 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tcvfn" Jul 07 15:51:48.945023 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:48.944991 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8"] Jul 07 15:51:48.948074 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:48.948054 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" Jul 07 15:51:48.950552 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:48.950523 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 15:51:48.950666 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:48.950528 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-w82p8\"" Jul 07 15:51:48.951541 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:48.951525 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 15:51:48.951634 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:48.951582 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 07 15:51:48.960305 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:48.960280 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8"] Jul 07 15:51:48.997767 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:48.997727 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnzxd\" (UniqueName: \"kubernetes.io/projected/15d1f05e-04bf-4925-8ef5-13d3665fa6bc-kube-api-access-lnzxd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8\" (UID: \"15d1f05e-04bf-4925-8ef5-13d3665fa6bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" Jul 07 15:51:48.997974 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:48.997810 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15d1f05e-04bf-4925-8ef5-13d3665fa6bc-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8\" (UID: \"15d1f05e-04bf-4925-8ef5-13d3665fa6bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" Jul 07 15:51:48.997974 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:48.997831 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15d1f05e-04bf-4925-8ef5-13d3665fa6bc-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8\" (UID: \"15d1f05e-04bf-4925-8ef5-13d3665fa6bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" Jul 07 15:51:48.997974 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:48.997855 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15d1f05e-04bf-4925-8ef5-13d3665fa6bc-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8\" (UID: \"15d1f05e-04bf-4925-8ef5-13d3665fa6bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" Jul 07 15:51:48.997974 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:48.997937 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15d1f05e-04bf-4925-8ef5-13d3665fa6bc-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8\" (UID: \"15d1f05e-04bf-4925-8ef5-13d3665fa6bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" Jul 07 15:51:48.997974 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:48.997965 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15d1f05e-04bf-4925-8ef5-13d3665fa6bc-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8\" (UID: \"15d1f05e-04bf-4925-8ef5-13d3665fa6bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" Jul 07 15:51:49.098881 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:49.098826 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnzxd\" (UniqueName: \"kubernetes.io/projected/15d1f05e-04bf-4925-8ef5-13d3665fa6bc-kube-api-access-lnzxd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8\" (UID: \"15d1f05e-04bf-4925-8ef5-13d3665fa6bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" Jul 07 15:51:49.099037 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:49.098928 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15d1f05e-04bf-4925-8ef5-13d3665fa6bc-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8\" (UID: \"15d1f05e-04bf-4925-8ef5-13d3665fa6bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" Jul 07 15:51:49.099037 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:49.098949 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15d1f05e-04bf-4925-8ef5-13d3665fa6bc-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8\" (UID: \"15d1f05e-04bf-4925-8ef5-13d3665fa6bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" Jul 07 15:51:49.099037 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:49.098972 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15d1f05e-04bf-4925-8ef5-13d3665fa6bc-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8\" (UID: \"15d1f05e-04bf-4925-8ef5-13d3665fa6bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" Jul 07 15:51:49.099037 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:49.098991 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15d1f05e-04bf-4925-8ef5-13d3665fa6bc-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8\" (UID: \"15d1f05e-04bf-4925-8ef5-13d3665fa6bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" Jul 07 15:51:49.099037 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:49.099010 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15d1f05e-04bf-4925-8ef5-13d3665fa6bc-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8\" (UID: \"15d1f05e-04bf-4925-8ef5-13d3665fa6bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" Jul 07 15:51:49.099447 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:49.099395 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15d1f05e-04bf-4925-8ef5-13d3665fa6bc-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8\" (UID: \"15d1f05e-04bf-4925-8ef5-13d3665fa6bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" Jul 07 15:51:49.099447 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:49.099411 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15d1f05e-04bf-4925-8ef5-13d3665fa6bc-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8\" (UID: \"15d1f05e-04bf-4925-8ef5-13d3665fa6bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" Jul 07 15:51:49.099638 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:49.099475 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15d1f05e-04bf-4925-8ef5-13d3665fa6bc-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8\" (UID: \"15d1f05e-04bf-4925-8ef5-13d3665fa6bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" Jul 07 15:51:49.101896 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:49.101847 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15d1f05e-04bf-4925-8ef5-13d3665fa6bc-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8\" (UID: \"15d1f05e-04bf-4925-8ef5-13d3665fa6bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" Jul 07 15:51:49.102020 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:49.101946 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15d1f05e-04bf-4925-8ef5-13d3665fa6bc-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8\" (UID: \"15d1f05e-04bf-4925-8ef5-13d3665fa6bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" Jul 07 15:51:49.108623 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:49.108593 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnzxd\" (UniqueName: \"kubernetes.io/projected/15d1f05e-04bf-4925-8ef5-13d3665fa6bc-kube-api-access-lnzxd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8\" (UID: \"15d1f05e-04bf-4925-8ef5-13d3665fa6bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" Jul 07 15:51:49.258317 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:49.258229 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" Jul 07 15:51:49.402634 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:49.402602 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8"] Jul 07 15:51:49.406660 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:51:49.406634 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15d1f05e_04bf_4925_8ef5_13d3665fa6bc.slice/crio-f13644fc015f20d611fdddf6059927165fea388bc9096ac0b9bef3ac30b910fa WatchSource:0}: Error finding container f13644fc015f20d611fdddf6059927165fea388bc9096ac0b9bef3ac30b910fa: Status 404 returned error can't find the container with id f13644fc015f20d611fdddf6059927165fea388bc9096ac0b9bef3ac30b910fa Jul 07 15:51:50.083278 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:50.083232 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" event={"ID":"15d1f05e-04bf-4925-8ef5-13d3665fa6bc","Type":"ContainerStarted","Data":"f13644fc015f20d611fdddf6059927165fea388bc9096ac0b9bef3ac30b910fa"} Jul 07 15:51:51.089328 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:51.089294 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" event={"ID":"15d1f05e-04bf-4925-8ef5-13d3665fa6bc","Type":"ContainerStarted","Data":"446b1f5fa10edea5e6212a70d08e3b9081ba1d146acfe303375bbe750c24748f"} Jul 07 15:51:51.110739 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:51.110690 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" podStartSLOduration=2.123835664 podStartE2EDuration="3.110674175s" totalImagesPullingTime="986.838511ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:51:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:51:51.10927175 +0000 UTC m=+598.204552869" watchObservedRunningTime="2026-07-07 15:51:51.110674175 +0000 UTC m=+598.205955254" Jul 07 15:51:53.341898 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:53.341857 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nfh8b_162f95df-a98f-49a6-a91e-689a8e22be46/console-operator/1.log" Jul 07 15:51:53.348137 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:53.348114 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4d86s_fbf78c29-89f3-4aad-82a7-2e8f99b6356e/ovn-acl-logging/0.log" Jul 07 15:51:53.351556 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:53.351535 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nfh8b_162f95df-a98f-49a6-a91e-689a8e22be46/console-operator/1.log" Jul 07 15:51:53.357402 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:53.357381 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4d86s_fbf78c29-89f3-4aad-82a7-2e8f99b6356e/ovn-acl-logging/0.log" Jul 07 15:51:55.645881 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:55.645835 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn"] Jul 07 15:51:55.648326 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:55.648309 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" Jul 07 15:51:55.650933 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:55.650915 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 07 15:51:55.663923 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:55.663895 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn"] Jul 07 15:51:55.759576 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:55.759526 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9b38285-3fcb-40f9-88c7-3541bbdc645f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2dwn\" (UID: \"d9b38285-3fcb-40f9-88c7-3541bbdc645f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" Jul 07 15:51:55.759787 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:55.759590 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9b38285-3fcb-40f9-88c7-3541bbdc645f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2dwn\" (UID: \"d9b38285-3fcb-40f9-88c7-3541bbdc645f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" Jul 07 15:51:55.759787 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:55.759657 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9b38285-3fcb-40f9-88c7-3541bbdc645f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2dwn\" (UID: \"d9b38285-3fcb-40f9-88c7-3541bbdc645f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" Jul 07 15:51:55.759787 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:55.759714 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9b38285-3fcb-40f9-88c7-3541bbdc645f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2dwn\" (UID: \"d9b38285-3fcb-40f9-88c7-3541bbdc645f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" Jul 07 15:51:55.759787 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:55.759756 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kncd\" (UniqueName: \"kubernetes.io/projected/d9b38285-3fcb-40f9-88c7-3541bbdc645f-kube-api-access-5kncd\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2dwn\" (UID: \"d9b38285-3fcb-40f9-88c7-3541bbdc645f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" Jul 07 15:51:55.759983 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:55.759803 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9b38285-3fcb-40f9-88c7-3541bbdc645f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2dwn\" (UID: \"d9b38285-3fcb-40f9-88c7-3541bbdc645f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" Jul 07 15:51:55.860474 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:55.860433 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9b38285-3fcb-40f9-88c7-3541bbdc645f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2dwn\" (UID: \"d9b38285-3fcb-40f9-88c7-3541bbdc645f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" Jul 07 15:51:55.860474 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:55.860478 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9b38285-3fcb-40f9-88c7-3541bbdc645f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2dwn\" (UID: \"d9b38285-3fcb-40f9-88c7-3541bbdc645f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" Jul 07 15:51:55.860713 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:55.860499 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9b38285-3fcb-40f9-88c7-3541bbdc645f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2dwn\" (UID: \"d9b38285-3fcb-40f9-88c7-3541bbdc645f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" Jul 07 15:51:55.860713 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:55.860517 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9b38285-3fcb-40f9-88c7-3541bbdc645f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2dwn\" (UID: \"d9b38285-3fcb-40f9-88c7-3541bbdc645f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" Jul 07 15:51:55.860713 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:55.860647 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kncd\" (UniqueName: \"kubernetes.io/projected/d9b38285-3fcb-40f9-88c7-3541bbdc645f-kube-api-access-5kncd\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2dwn\" (UID: \"d9b38285-3fcb-40f9-88c7-3541bbdc645f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" Jul 07 15:51:55.860896 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:55.860729 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9b38285-3fcb-40f9-88c7-3541bbdc645f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2dwn\" (UID: \"d9b38285-3fcb-40f9-88c7-3541bbdc645f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" Jul 07 15:51:55.860896 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:55.860846 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9b38285-3fcb-40f9-88c7-3541bbdc645f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2dwn\" (UID: \"d9b38285-3fcb-40f9-88c7-3541bbdc645f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" Jul 07 15:51:55.860994 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:55.860960 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9b38285-3fcb-40f9-88c7-3541bbdc645f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2dwn\" (UID: \"d9b38285-3fcb-40f9-88c7-3541bbdc645f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" Jul 07 15:51:55.861146 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:55.861121 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9b38285-3fcb-40f9-88c7-3541bbdc645f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2dwn\" (UID: \"d9b38285-3fcb-40f9-88c7-3541bbdc645f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" Jul 07 15:51:55.862748 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:55.862720 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9b38285-3fcb-40f9-88c7-3541bbdc645f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2dwn\" (UID: \"d9b38285-3fcb-40f9-88c7-3541bbdc645f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" Jul 07 15:51:55.863001 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:55.862983 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9b38285-3fcb-40f9-88c7-3541bbdc645f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2dwn\" (UID: \"d9b38285-3fcb-40f9-88c7-3541bbdc645f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" Jul 07 15:51:55.871657 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:55.871639 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kncd\" (UniqueName: \"kubernetes.io/projected/d9b38285-3fcb-40f9-88c7-3541bbdc645f-kube-api-access-5kncd\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2dwn\" (UID: \"d9b38285-3fcb-40f9-88c7-3541bbdc645f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" Jul 07 15:51:55.958428 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:55.958333 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" Jul 07 15:51:56.106053 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:56.106022 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn"] Jul 07 15:51:56.107482 ip-10-0-131-221 kubenswrapper[2575]: W0707 15:51:56.107458 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9b38285_3fcb_40f9_88c7_3541bbdc645f.slice/crio-0b033faf6308581802006178575e4fa7287ab0bdd53ba7c80c81c56b3a4b5706 WatchSource:0}: Error finding container 0b033faf6308581802006178575e4fa7287ab0bdd53ba7c80c81c56b3a4b5706: Status 404 returned error can't find the container with id 0b033faf6308581802006178575e4fa7287ab0bdd53ba7c80c81c56b3a4b5706 Jul 07 15:51:56.109259 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:56.109242 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 15:51:56.120733 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:56.119704 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" event={"ID":"d9b38285-3fcb-40f9-88c7-3541bbdc645f","Type":"ContainerStarted","Data":"0b033faf6308581802006178575e4fa7287ab0bdd53ba7c80c81c56b3a4b5706"} Jul 07 15:51:57.126575 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:57.126532 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" event={"ID":"d9b38285-3fcb-40f9-88c7-3541bbdc645f","Type":"ContainerStarted","Data":"f0d12a9fafd25b9ebe8d74c967d69e146394d2951191342026f38e98c981a225"} Jul 07 15:51:57.150091 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:57.150038 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" podStartSLOduration=1.934982734 podStartE2EDuration="2.150022318s" totalImagesPullingTime="215.039584ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:51:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:51:57.147910708 +0000 UTC m=+604.243191790" watchObservedRunningTime="2026-07-07 15:51:57.150022318 +0000 UTC m=+604.245303396" Jul 07 15:51:59.258548 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:59.258507 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" Jul 07 15:51:59.258970 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:59.258581 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" Jul 07 15:51:59.271044 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:51:59.271022 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" Jul 07 15:52:00.152484 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:52:00.152449 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8" Jul 07 15:52:05.959114 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:52:05.959062 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" Jul 07 15:52:05.959653 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:52:05.959129 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" Jul 07 15:52:05.971416 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:52:05.971389 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" Jul 07 15:52:06.180023 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:52:06.179989 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2dwn" Jul 07 15:56:53.372882 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:56:53.372830 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nfh8b_162f95df-a98f-49a6-a91e-689a8e22be46/console-operator/1.log" Jul 07 15:56:53.379422 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:56:53.379400 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4d86s_fbf78c29-89f3-4aad-82a7-2e8f99b6356e/ovn-acl-logging/0.log" Jul 07 15:56:53.379727 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:56:53.379709 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nfh8b_162f95df-a98f-49a6-a91e-689a8e22be46/console-operator/1.log" Jul 07 15:56:53.386503 ip-10-0-131-221 kubenswrapper[2575]: I0707 15:56:53.386485 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4d86s_fbf78c29-89f3-4aad-82a7-2e8f99b6356e/ovn-acl-logging/0.log" Jul 07 16:00:00.168881 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:00:00.168828 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29724000-pbjkx"] Jul 07 16:00:00.171909 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:00:00.171892 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724000-pbjkx" Jul 07 16:00:00.174970 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:00:00.174948 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-5vqnq\"" Jul 07 16:00:00.182928 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:00:00.182907 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29724000-pbjkx"] Jul 07 16:00:00.270945 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:00:00.270899 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7g96\" (UniqueName: \"kubernetes.io/projected/1a3d17ae-cd57-4f29-9182-f21a38c4c8e4-kube-api-access-x7g96\") pod \"maas-api-key-cleanup-29724000-pbjkx\" (UID: \"1a3d17ae-cd57-4f29-9182-f21a38c4c8e4\") " pod="opendatahub/maas-api-key-cleanup-29724000-pbjkx" Jul 07 16:00:00.371416 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:00:00.371378 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7g96\" (UniqueName: \"kubernetes.io/projected/1a3d17ae-cd57-4f29-9182-f21a38c4c8e4-kube-api-access-x7g96\") pod \"maas-api-key-cleanup-29724000-pbjkx\" (UID: \"1a3d17ae-cd57-4f29-9182-f21a38c4c8e4\") " pod="opendatahub/maas-api-key-cleanup-29724000-pbjkx" Jul 07 16:00:00.385312 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:00:00.385275 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7g96\" (UniqueName: \"kubernetes.io/projected/1a3d17ae-cd57-4f29-9182-f21a38c4c8e4-kube-api-access-x7g96\") pod \"maas-api-key-cleanup-29724000-pbjkx\" (UID: \"1a3d17ae-cd57-4f29-9182-f21a38c4c8e4\") " pod="opendatahub/maas-api-key-cleanup-29724000-pbjkx" Jul 07 16:00:00.482271 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:00:00.482180 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724000-pbjkx" Jul 07 16:00:00.621975 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:00:00.621945 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29724000-pbjkx"] Jul 07 16:00:00.624400 ip-10-0-131-221 kubenswrapper[2575]: W0707 16:00:00.624353 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a3d17ae_cd57_4f29_9182_f21a38c4c8e4.slice/crio-e84df2302426608dc2b80abbce12ee9237ba3d96cfe337c7781d71afbedc4d62 WatchSource:0}: Error finding container e84df2302426608dc2b80abbce12ee9237ba3d96cfe337c7781d71afbedc4d62: Status 404 returned error can't find the container with id e84df2302426608dc2b80abbce12ee9237ba3d96cfe337c7781d71afbedc4d62 Jul 07 16:00:00.626302 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:00:00.626284 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 16:00:01.338490 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:00:01.338450 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724000-pbjkx" event={"ID":"1a3d17ae-cd57-4f29-9182-f21a38c4c8e4","Type":"ContainerStarted","Data":"e84df2302426608dc2b80abbce12ee9237ba3d96cfe337c7781d71afbedc4d62"} Jul 07 16:00:05.358447 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:00:05.358411 2575 generic.go:356] "Generic (PLEG): container finished" podID="1a3d17ae-cd57-4f29-9182-f21a38c4c8e4" containerID="d6cf3b2947fd197fba280475021b5f604e846033670f6231f3749fc9fdf08907" exitCode=0 Jul 07 16:00:05.358833 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:00:05.358502 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724000-pbjkx" event={"ID":"1a3d17ae-cd57-4f29-9182-f21a38c4c8e4","Type":"ContainerDied","Data":"d6cf3b2947fd197fba280475021b5f604e846033670f6231f3749fc9fdf08907"} Jul 07 16:00:06.494255 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:00:06.494233 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724000-pbjkx" Jul 07 16:00:06.629133 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:00:06.629048 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7g96\" (UniqueName: \"kubernetes.io/projected/1a3d17ae-cd57-4f29-9182-f21a38c4c8e4-kube-api-access-x7g96\") pod \"1a3d17ae-cd57-4f29-9182-f21a38c4c8e4\" (UID: \"1a3d17ae-cd57-4f29-9182-f21a38c4c8e4\") " Jul 07 16:00:06.631197 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:00:06.631172 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a3d17ae-cd57-4f29-9182-f21a38c4c8e4-kube-api-access-x7g96" (OuterVolumeSpecName: "kube-api-access-x7g96") pod "1a3d17ae-cd57-4f29-9182-f21a38c4c8e4" (UID: "1a3d17ae-cd57-4f29-9182-f21a38c4c8e4"). InnerVolumeSpecName "kube-api-access-x7g96". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:00:06.729758 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:00:06.729721 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7g96\" (UniqueName: \"kubernetes.io/projected/1a3d17ae-cd57-4f29-9182-f21a38c4c8e4-kube-api-access-x7g96\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:00:07.369594 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:00:07.369561 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724000-pbjkx" event={"ID":"1a3d17ae-cd57-4f29-9182-f21a38c4c8e4","Type":"ContainerDied","Data":"e84df2302426608dc2b80abbce12ee9237ba3d96cfe337c7781d71afbedc4d62"} Jul 07 16:00:07.369594 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:00:07.369583 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724000-pbjkx" Jul 07 16:00:07.369809 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:00:07.369607 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e84df2302426608dc2b80abbce12ee9237ba3d96cfe337c7781d71afbedc4d62" Jul 07 16:01:53.403694 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:01:53.403617 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nfh8b_162f95df-a98f-49a6-a91e-689a8e22be46/console-operator/1.log" Jul 07 16:01:53.407542 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:01:53.407519 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nfh8b_162f95df-a98f-49a6-a91e-689a8e22be46/console-operator/1.log" Jul 07 16:01:53.410285 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:01:53.410266 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4d86s_fbf78c29-89f3-4aad-82a7-2e8f99b6356e/ovn-acl-logging/0.log" Jul 07 16:01:53.413932 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:01:53.413912 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4d86s_fbf78c29-89f3-4aad-82a7-2e8f99b6356e/ovn-acl-logging/0.log" Jul 07 16:06:53.436089 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:06:53.436064 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nfh8b_162f95df-a98f-49a6-a91e-689a8e22be46/console-operator/1.log" Jul 07 16:06:53.436726 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:06:53.436707 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nfh8b_162f95df-a98f-49a6-a91e-689a8e22be46/console-operator/1.log" Jul 07 16:06:53.442397 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:06:53.442377 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4d86s_fbf78c29-89f3-4aad-82a7-2e8f99b6356e/ovn-acl-logging/0.log" Jul 07 16:06:53.442807 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:06:53.442793 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4d86s_fbf78c29-89f3-4aad-82a7-2e8f99b6356e/ovn-acl-logging/0.log" Jul 07 16:07:13.848468 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:13.848435 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf"] Jul 07 16:07:13.848852 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:13.848805 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a3d17ae-cd57-4f29-9182-f21a38c4c8e4" containerName="cleanup" Jul 07 16:07:13.848852 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:13.848817 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a3d17ae-cd57-4f29-9182-f21a38c4c8e4" containerName="cleanup" Jul 07 16:07:13.848949 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:13.848931 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a3d17ae-cd57-4f29-9182-f21a38c4c8e4" containerName="cleanup" Jul 07 16:07:13.851561 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:13.851538 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf" Jul 07 16:07:13.854072 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:13.854048 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-ac5352d4-serving-cert\"" Jul 07 16:07:13.854204 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:13.854126 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-5vqnq\"" Jul 07 16:07:13.863565 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:13.863539 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf"] Jul 07 16:07:13.890505 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:13.890469 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bad8f442-f93b-4d85-a1b4-508e6ddfe152-openshift-service-ca\") pod \"maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf\" (UID: \"bad8f442-f93b-4d85-a1b4-508e6ddfe152\") " pod="opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf" Jul 07 16:07:13.890666 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:13.890570 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bad8f442-f93b-4d85-a1b4-508e6ddfe152-maas-api-tls\") pod \"maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf\" (UID: \"bad8f442-f93b-4d85-a1b4-508e6ddfe152\") " pod="opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf" Jul 07 16:07:13.890666 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:13.890599 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlm6h\" (UniqueName: \"kubernetes.io/projected/bad8f442-f93b-4d85-a1b4-508e6ddfe152-kube-api-access-rlm6h\") pod \"maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf\" (UID: \"bad8f442-f93b-4d85-a1b4-508e6ddfe152\") " pod="opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf" Jul 07 16:07:13.991885 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:13.991823 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bad8f442-f93b-4d85-a1b4-508e6ddfe152-openshift-service-ca\") pod \"maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf\" (UID: \"bad8f442-f93b-4d85-a1b4-508e6ddfe152\") " pod="opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf" Jul 07 16:07:13.992093 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:13.991926 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bad8f442-f93b-4d85-a1b4-508e6ddfe152-maas-api-tls\") pod \"maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf\" (UID: \"bad8f442-f93b-4d85-a1b4-508e6ddfe152\") " pod="opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf" Jul 07 16:07:13.992093 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:13.991947 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlm6h\" (UniqueName: \"kubernetes.io/projected/bad8f442-f93b-4d85-a1b4-508e6ddfe152-kube-api-access-rlm6h\") pod \"maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf\" (UID: \"bad8f442-f93b-4d85-a1b4-508e6ddfe152\") " pod="opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf" Jul 07 16:07:13.992547 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:13.992517 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bad8f442-f93b-4d85-a1b4-508e6ddfe152-openshift-service-ca\") pod \"maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf\" (UID: \"bad8f442-f93b-4d85-a1b4-508e6ddfe152\") " pod="opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf" Jul 07 16:07:13.994337 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:13.994315 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bad8f442-f93b-4d85-a1b4-508e6ddfe152-maas-api-tls\") pod \"maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf\" (UID: \"bad8f442-f93b-4d85-a1b4-508e6ddfe152\") " pod="opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf" Jul 07 16:07:14.006390 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:14.006364 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlm6h\" (UniqueName: \"kubernetes.io/projected/bad8f442-f93b-4d85-a1b4-508e6ddfe152-kube-api-access-rlm6h\") pod \"maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf\" (UID: \"bad8f442-f93b-4d85-a1b4-508e6ddfe152\") " pod="opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf" Jul 07 16:07:14.165633 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:14.165542 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf" Jul 07 16:07:14.307985 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:14.307949 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf"] Jul 07 16:07:14.315940 ip-10-0-131-221 kubenswrapper[2575]: W0707 16:07:14.315911 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbad8f442_f93b_4d85_a1b4_508e6ddfe152.slice/crio-e7c7df8f18bde31d0f3211aca829ff23d5ff0b286488a00cbc974e5c20fe9843 WatchSource:0}: Error finding container e7c7df8f18bde31d0f3211aca829ff23d5ff0b286488a00cbc974e5c20fe9843: Status 404 returned error can't find the container with id e7c7df8f18bde31d0f3211aca829ff23d5ff0b286488a00cbc974e5c20fe9843 Jul 07 16:07:14.317444 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:14.317424 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 16:07:14.328661 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:14.328611 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf" event={"ID":"bad8f442-f93b-4d85-a1b4-508e6ddfe152","Type":"ContainerStarted","Data":"e7c7df8f18bde31d0f3211aca829ff23d5ff0b286488a00cbc974e5c20fe9843"} Jul 07 16:07:17.345639 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:17.345606 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf" event={"ID":"bad8f442-f93b-4d85-a1b4-508e6ddfe152","Type":"ContainerStarted","Data":"6466a0f02025fd57b65232e3bf680b5357cdae641f4eb9bbb2f6cc32846d8b8a"} Jul 07 16:07:17.346054 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:17.345729 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf" Jul 07 16:07:17.368593 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:17.368546 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf" podStartSLOduration=2.370848619 podStartE2EDuration="4.36853136s" totalImagesPullingTime="1.997682741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:07:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:07:17.367973325 +0000 UTC m=+1524.463254405" watchObservedRunningTime="2026-07-07 16:07:17.36853136 +0000 UTC m=+1524.463812488" Jul 07 16:07:19.348434 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:19.348386 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf"] Jul 07 16:07:19.356247 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:19.356196 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf" podUID="bad8f442-f93b-4d85-a1b4-508e6ddfe152" containerName="maas-api" containerID="cri-o://6466a0f02025fd57b65232e3bf680b5357cdae641f4eb9bbb2f6cc32846d8b8a" gracePeriod=30 Jul 07 16:07:19.599622 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:19.599570 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf" Jul 07 16:07:19.648105 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:19.648076 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bad8f442-f93b-4d85-a1b4-508e6ddfe152-openshift-service-ca\") pod \"bad8f442-f93b-4d85-a1b4-508e6ddfe152\" (UID: \"bad8f442-f93b-4d85-a1b4-508e6ddfe152\") " Jul 07 16:07:19.648270 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:19.648134 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bad8f442-f93b-4d85-a1b4-508e6ddfe152-maas-api-tls\") pod \"bad8f442-f93b-4d85-a1b4-508e6ddfe152\" (UID: \"bad8f442-f93b-4d85-a1b4-508e6ddfe152\") " Jul 07 16:07:19.648270 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:19.648205 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlm6h\" (UniqueName: \"kubernetes.io/projected/bad8f442-f93b-4d85-a1b4-508e6ddfe152-kube-api-access-rlm6h\") pod \"bad8f442-f93b-4d85-a1b4-508e6ddfe152\" (UID: \"bad8f442-f93b-4d85-a1b4-508e6ddfe152\") " Jul 07 16:07:19.648513 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:19.648484 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bad8f442-f93b-4d85-a1b4-508e6ddfe152-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bad8f442-f93b-4d85-a1b4-508e6ddfe152" (UID: "bad8f442-f93b-4d85-a1b4-508e6ddfe152"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:07:19.650323 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:19.650299 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bad8f442-f93b-4d85-a1b4-508e6ddfe152-kube-api-access-rlm6h" (OuterVolumeSpecName: "kube-api-access-rlm6h") pod "bad8f442-f93b-4d85-a1b4-508e6ddfe152" (UID: "bad8f442-f93b-4d85-a1b4-508e6ddfe152"). InnerVolumeSpecName "kube-api-access-rlm6h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:07:19.650398 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:19.650322 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bad8f442-f93b-4d85-a1b4-508e6ddfe152-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bad8f442-f93b-4d85-a1b4-508e6ddfe152" (UID: "bad8f442-f93b-4d85-a1b4-508e6ddfe152"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:07:19.749556 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:19.749517 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bad8f442-f93b-4d85-a1b4-508e6ddfe152-maas-api-tls\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:07:19.749556 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:19.749549 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlm6h\" (UniqueName: \"kubernetes.io/projected/bad8f442-f93b-4d85-a1b4-508e6ddfe152-kube-api-access-rlm6h\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:07:19.749556 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:19.749563 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bad8f442-f93b-4d85-a1b4-508e6ddfe152-openshift-service-ca\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:07:20.362402 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:20.362365 2575 generic.go:356] "Generic (PLEG): container finished" podID="bad8f442-f93b-4d85-a1b4-508e6ddfe152" containerID="6466a0f02025fd57b65232e3bf680b5357cdae641f4eb9bbb2f6cc32846d8b8a" exitCode=0 Jul 07 16:07:20.362795 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:20.362414 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf" Jul 07 16:07:20.362795 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:20.362422 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf" event={"ID":"bad8f442-f93b-4d85-a1b4-508e6ddfe152","Type":"ContainerDied","Data":"6466a0f02025fd57b65232e3bf680b5357cdae641f4eb9bbb2f6cc32846d8b8a"} Jul 07 16:07:20.362795 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:20.362470 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf" event={"ID":"bad8f442-f93b-4d85-a1b4-508e6ddfe152","Type":"ContainerDied","Data":"e7c7df8f18bde31d0f3211aca829ff23d5ff0b286488a00cbc974e5c20fe9843"} Jul 07 16:07:20.362795 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:20.362493 2575 scope.go:117] "RemoveContainer" containerID="6466a0f02025fd57b65232e3bf680b5357cdae641f4eb9bbb2f6cc32846d8b8a" Jul 07 16:07:20.372092 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:20.372072 2575 scope.go:117] "RemoveContainer" containerID="6466a0f02025fd57b65232e3bf680b5357cdae641f4eb9bbb2f6cc32846d8b8a" Jul 07 16:07:20.372478 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:07:20.372439 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6466a0f02025fd57b65232e3bf680b5357cdae641f4eb9bbb2f6cc32846d8b8a\": container with ID starting with 6466a0f02025fd57b65232e3bf680b5357cdae641f4eb9bbb2f6cc32846d8b8a not found: ID does not exist" containerID="6466a0f02025fd57b65232e3bf680b5357cdae641f4eb9bbb2f6cc32846d8b8a" Jul 07 16:07:20.372551 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:20.372479 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6466a0f02025fd57b65232e3bf680b5357cdae641f4eb9bbb2f6cc32846d8b8a"} err="failed to get container status \"6466a0f02025fd57b65232e3bf680b5357cdae641f4eb9bbb2f6cc32846d8b8a\": rpc error: code = NotFound desc = could not find container \"6466a0f02025fd57b65232e3bf680b5357cdae641f4eb9bbb2f6cc32846d8b8a\": container with ID starting with 6466a0f02025fd57b65232e3bf680b5357cdae641f4eb9bbb2f6cc32846d8b8a not found: ID does not exist" Jul 07 16:07:20.392227 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:20.392172 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf"] Jul 07 16:07:20.394123 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:20.394094 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-ac5352d4-5fbf8fb666-cpwjf"] Jul 07 16:07:21.436022 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:21.435991 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bad8f442-f93b-4d85-a1b4-508e6ddfe152" path="/var/lib/kubelet/pods/bad8f442-f93b-4d85-a1b4-508e6ddfe152/volumes" Jul 07 16:07:26.283585 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.283551 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h"] Jul 07 16:07:26.284059 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.283938 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bad8f442-f93b-4d85-a1b4-508e6ddfe152" containerName="maas-api" Jul 07 16:07:26.284059 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.283951 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bad8f442-f93b-4d85-a1b4-508e6ddfe152" containerName="maas-api" Jul 07 16:07:26.284059 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.284024 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="bad8f442-f93b-4d85-a1b4-508e6ddfe152" containerName="maas-api" Jul 07 16:07:26.286514 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.286496 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.291828 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.291806 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b203da5c-openshift-default-dockercfg-fldnz\"" Jul 07 16:07:26.309758 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.309727 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h"] Jul 07 16:07:26.407974 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.407935 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istio-envoy\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.408153 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.407981 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istio-token\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.408153 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.408037 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-credential-socket\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.408153 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.408099 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-workload-certs\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.408153 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.408120 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-workload-socket\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.408153 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.408142 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kdv8\" (UniqueName: \"kubernetes.io/projected/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-kube-api-access-5kdv8\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.408347 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.408235 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istiod-ca-cert\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.408347 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.408283 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istio-data\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.408347 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.408308 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istio-podinfo\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.509613 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.509579 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-workload-certs\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.509613 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.509615 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-workload-socket\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.509911 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.509633 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kdv8\" (UniqueName: \"kubernetes.io/projected/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-kube-api-access-5kdv8\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.509911 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.509702 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istiod-ca-cert\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.509911 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.509765 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istio-data\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.509911 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.509796 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istio-podinfo\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.509911 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.509835 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istio-envoy\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.509911 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.509897 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istio-token\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.510221 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.509931 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-credential-socket\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.510221 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.510056 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-workload-certs\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.510221 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.510143 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-workload-socket\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.510407 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.510322 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-credential-socket\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.510407 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.510382 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istio-data\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.510545 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.510525 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istiod-ca-cert\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.512280 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.512260 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istio-envoy\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.512385 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.512367 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istio-podinfo\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.520137 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.520111 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istio-token\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.520399 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.520382 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kdv8\" (UniqueName: \"kubernetes.io/projected/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-kube-api-access-5kdv8\") pod \"e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.597666 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.597632 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:26.738775 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:26.738728 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h"] Jul 07 16:07:26.741190 ip-10-0-131-221 kubenswrapper[2575]: W0707 16:07:26.741166 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b882148_c0e8_4e0b_bc16_9aa88e6e7f05.slice/crio-fe096f59e8c3506dea21a3a519a0c47d6f3737bb039f1eec604d967c5a5da53d WatchSource:0}: Error finding container fe096f59e8c3506dea21a3a519a0c47d6f3737bb039f1eec604d967c5a5da53d: Status 404 returned error can't find the container with id fe096f59e8c3506dea21a3a519a0c47d6f3737bb039f1eec604d967c5a5da53d Jul 07 16:07:27.396840 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:27.396805 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" event={"ID":"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05","Type":"ContainerStarted","Data":"fe096f59e8c3506dea21a3a519a0c47d6f3737bb039f1eec604d967c5a5da53d"} Jul 07 16:07:27.659270 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:27.659181 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x"] Jul 07 16:07:27.665110 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:27.665089 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x" Jul 07 16:07:27.668765 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:27.668740 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-b203da5c-serving-cert\"" Jul 07 16:07:27.668916 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:27.668822 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-5vqnq\"" Jul 07 16:07:27.678450 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:27.678427 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x"] Jul 07 16:07:27.722850 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:27.722818 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f66871d-0b81-484d-9556-48b025f7cfe1-maas-api-tls\") pod \"maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x\" (UID: \"4f66871d-0b81-484d-9556-48b025f7cfe1\") " pod="opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x" Jul 07 16:07:27.723031 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:27.722856 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f66871d-0b81-484d-9556-48b025f7cfe1-openshift-service-ca\") pod \"maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x\" (UID: \"4f66871d-0b81-484d-9556-48b025f7cfe1\") " pod="opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x" Jul 07 16:07:27.723031 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:27.722941 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxk84\" (UniqueName: \"kubernetes.io/projected/4f66871d-0b81-484d-9556-48b025f7cfe1-kube-api-access-fxk84\") pod \"maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x\" (UID: \"4f66871d-0b81-484d-9556-48b025f7cfe1\") " pod="opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x" Jul 07 16:07:27.824003 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:27.823967 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f66871d-0b81-484d-9556-48b025f7cfe1-maas-api-tls\") pod \"maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x\" (UID: \"4f66871d-0b81-484d-9556-48b025f7cfe1\") " pod="opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x" Jul 07 16:07:27.824190 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:27.824010 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f66871d-0b81-484d-9556-48b025f7cfe1-openshift-service-ca\") pod \"maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x\" (UID: \"4f66871d-0b81-484d-9556-48b025f7cfe1\") " pod="opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x" Jul 07 16:07:27.824190 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:27.824066 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxk84\" (UniqueName: \"kubernetes.io/projected/4f66871d-0b81-484d-9556-48b025f7cfe1-kube-api-access-fxk84\") pod \"maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x\" (UID: \"4f66871d-0b81-484d-9556-48b025f7cfe1\") " pod="opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x" Jul 07 16:07:27.824807 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:27.824774 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f66871d-0b81-484d-9556-48b025f7cfe1-openshift-service-ca\") pod \"maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x\" (UID: \"4f66871d-0b81-484d-9556-48b025f7cfe1\") " pod="opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x" Jul 07 16:07:27.826954 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:27.826932 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f66871d-0b81-484d-9556-48b025f7cfe1-maas-api-tls\") pod \"maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x\" (UID: \"4f66871d-0b81-484d-9556-48b025f7cfe1\") " pod="opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x" Jul 07 16:07:27.860058 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:27.860002 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxk84\" (UniqueName: \"kubernetes.io/projected/4f66871d-0b81-484d-9556-48b025f7cfe1-kube-api-access-fxk84\") pod \"maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x\" (UID: \"4f66871d-0b81-484d-9556-48b025f7cfe1\") " pod="opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x" Jul 07 16:07:27.977676 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:27.977597 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x" Jul 07 16:07:28.136245 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:28.136212 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x"] Jul 07 16:07:28.138014 ip-10-0-131-221 kubenswrapper[2575]: W0707 16:07:28.137980 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f66871d_0b81_484d_9556_48b025f7cfe1.slice/crio-7da45fa9efae6af0616f68a88eeb9c8cb7d78cc5f6df8e5e333c25065b9ff982 WatchSource:0}: Error finding container 7da45fa9efae6af0616f68a88eeb9c8cb7d78cc5f6df8e5e333c25065b9ff982: Status 404 returned error can't find the container with id 7da45fa9efae6af0616f68a88eeb9c8cb7d78cc5f6df8e5e333c25065b9ff982 Jul 07 16:07:28.409058 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:28.409017 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x" event={"ID":"4f66871d-0b81-484d-9556-48b025f7cfe1","Type":"ContainerStarted","Data":"7da45fa9efae6af0616f68a88eeb9c8cb7d78cc5f6df8e5e333c25065b9ff982"} Jul 07 16:07:29.416993 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:29.416958 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x" event={"ID":"4f66871d-0b81-484d-9556-48b025f7cfe1","Type":"ContainerStarted","Data":"73769e75afc074fa685911649cd4083da12d715418321e81cf151fd2bdbf8b7a"} Jul 07 16:07:29.417446 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:29.417105 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x" Jul 07 16:07:29.449752 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:29.449693 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x" podStartSLOduration=2.049294311 podStartE2EDuration="2.449671953s" totalImagesPullingTime="400.377642ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:07:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:07:29.446815565 +0000 UTC m=+1536.542096646" watchObservedRunningTime="2026-07-07 16:07:29.449671953 +0000 UTC m=+1536.544953017" Jul 07 16:07:30.105632 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:30.105592 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:07:30.105747 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:30.105675 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:07:30.105747 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:30.105704 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:07:30.423536 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:30.423443 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" event={"ID":"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05","Type":"ContainerStarted","Data":"760ada191c13c06518e3dab916bbe80f8a53a1f4913941069ef62dfb9269ce54"} Jul 07 16:07:30.451423 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:30.451371 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" podStartSLOduration=1.089655106 podStartE2EDuration="4.451354408s" totalImagesPullingTime="3.361699302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:07:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:07:30.449163621 +0000 UTC m=+1537.544444736" watchObservedRunningTime="2026-07-07 16:07:30.451354408 +0000 UTC m=+1537.546635486" Jul 07 16:07:30.598507 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:30.598475 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:31.602405 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:31.602374 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:32.433011 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:32.432983 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:32.439819 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:32.439796 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:33.118557 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:33.118526 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x"] Jul 07 16:07:33.118997 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:33.118692 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x" podUID="4f66871d-0b81-484d-9556-48b025f7cfe1" containerName="maas-api" containerID="cri-o://73769e75afc074fa685911649cd4083da12d715418321e81cf151fd2bdbf8b7a" gracePeriod=30 Jul 07 16:07:33.123560 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:33.123539 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x" Jul 07 16:07:33.412961 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:33.412933 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x" Jul 07 16:07:33.473115 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:33.473081 2575 generic.go:356] "Generic (PLEG): container finished" podID="4f66871d-0b81-484d-9556-48b025f7cfe1" containerID="73769e75afc074fa685911649cd4083da12d715418321e81cf151fd2bdbf8b7a" exitCode=0 Jul 07 16:07:33.473322 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:33.473250 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x" Jul 07 16:07:33.475116 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:33.475091 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x" event={"ID":"4f66871d-0b81-484d-9556-48b025f7cfe1","Type":"ContainerDied","Data":"73769e75afc074fa685911649cd4083da12d715418321e81cf151fd2bdbf8b7a"} Jul 07 16:07:33.475244 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:33.475200 2575 scope.go:117] "RemoveContainer" containerID="73769e75afc074fa685911649cd4083da12d715418321e81cf151fd2bdbf8b7a" Jul 07 16:07:33.475395 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:33.475131 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x" event={"ID":"4f66871d-0b81-484d-9556-48b025f7cfe1","Type":"ContainerDied","Data":"7da45fa9efae6af0616f68a88eeb9c8cb7d78cc5f6df8e5e333c25065b9ff982"} Jul 07 16:07:33.484013 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:33.483995 2575 scope.go:117] "RemoveContainer" containerID="73769e75afc074fa685911649cd4083da12d715418321e81cf151fd2bdbf8b7a" Jul 07 16:07:33.484291 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:07:33.484267 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73769e75afc074fa685911649cd4083da12d715418321e81cf151fd2bdbf8b7a\": container with ID starting with 73769e75afc074fa685911649cd4083da12d715418321e81cf151fd2bdbf8b7a not found: ID does not exist" containerID="73769e75afc074fa685911649cd4083da12d715418321e81cf151fd2bdbf8b7a" Jul 07 16:07:33.484386 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:33.484298 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73769e75afc074fa685911649cd4083da12d715418321e81cf151fd2bdbf8b7a"} err="failed to get container status \"73769e75afc074fa685911649cd4083da12d715418321e81cf151fd2bdbf8b7a\": rpc error: code = NotFound desc = could not find container \"73769e75afc074fa685911649cd4083da12d715418321e81cf151fd2bdbf8b7a\": container with ID starting with 73769e75afc074fa685911649cd4083da12d715418321e81cf151fd2bdbf8b7a not found: ID does not exist" Jul 07 16:07:33.582855 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:33.582828 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxk84\" (UniqueName: \"kubernetes.io/projected/4f66871d-0b81-484d-9556-48b025f7cfe1-kube-api-access-fxk84\") pod \"4f66871d-0b81-484d-9556-48b025f7cfe1\" (UID: \"4f66871d-0b81-484d-9556-48b025f7cfe1\") " Jul 07 16:07:33.583008 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:33.582918 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f66871d-0b81-484d-9556-48b025f7cfe1-maas-api-tls\") pod \"4f66871d-0b81-484d-9556-48b025f7cfe1\" (UID: \"4f66871d-0b81-484d-9556-48b025f7cfe1\") " Jul 07 16:07:33.583008 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:33.582966 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f66871d-0b81-484d-9556-48b025f7cfe1-openshift-service-ca\") pod \"4f66871d-0b81-484d-9556-48b025f7cfe1\" (UID: \"4f66871d-0b81-484d-9556-48b025f7cfe1\") " Jul 07 16:07:33.584180 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:33.583306 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f66871d-0b81-484d-9556-48b025f7cfe1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4f66871d-0b81-484d-9556-48b025f7cfe1" (UID: "4f66871d-0b81-484d-9556-48b025f7cfe1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:07:33.584180 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:33.584100 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f66871d-0b81-484d-9556-48b025f7cfe1-openshift-service-ca\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:07:33.585015 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:33.584996 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f66871d-0b81-484d-9556-48b025f7cfe1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4f66871d-0b81-484d-9556-48b025f7cfe1" (UID: "4f66871d-0b81-484d-9556-48b025f7cfe1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:07:33.585131 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:33.585112 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f66871d-0b81-484d-9556-48b025f7cfe1-kube-api-access-fxk84" (OuterVolumeSpecName: "kube-api-access-fxk84") pod "4f66871d-0b81-484d-9556-48b025f7cfe1" (UID: "4f66871d-0b81-484d-9556-48b025f7cfe1"). InnerVolumeSpecName "kube-api-access-fxk84". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:07:33.685181 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:33.685091 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxk84\" (UniqueName: \"kubernetes.io/projected/4f66871d-0b81-484d-9556-48b025f7cfe1-kube-api-access-fxk84\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:07:33.685181 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:33.685121 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f66871d-0b81-484d-9556-48b025f7cfe1-maas-api-tls\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:07:33.797603 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:33.797575 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x"] Jul 07 16:07:33.802216 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:33.802192 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-b203da5c-5d9f49d44f-mtl7x"] Jul 07 16:07:35.436003 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:35.435969 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f66871d-0b81-484d-9556-48b025f7cfe1" path="/var/lib/kubelet/pods/4f66871d-0b81-484d-9556-48b025f7cfe1/volumes" Jul 07 16:07:38.840252 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:38.840216 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h"] Jul 07 16:07:38.840700 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:38.840454 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" podUID="3b882148-c0e8-4e0b-bc16-9aa88e6e7f05" containerName="istio-proxy" containerID="cri-o://760ada191c13c06518e3dab916bbe80f8a53a1f4913941069ef62dfb9269ce54" gracePeriod=30 Jul 07 16:07:38.982014 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:38.981989 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:39.136113 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.136080 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istio-envoy\") pod \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " Jul 07 16:07:39.136113 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.136118 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kdv8\" (UniqueName: \"kubernetes.io/projected/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-kube-api-access-5kdv8\") pod \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " Jul 07 16:07:39.136345 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.136161 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istio-data\") pod \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " Jul 07 16:07:39.136345 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.136178 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-workload-socket\") pod \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " Jul 07 16:07:39.136345 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.136205 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istio-podinfo\") pod \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " Jul 07 16:07:39.136345 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.136257 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-credential-socket\") pod \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " Jul 07 16:07:39.136345 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.136281 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-workload-certs\") pod \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " Jul 07 16:07:39.136345 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.136317 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istiod-ca-cert\") pod \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " Jul 07 16:07:39.136648 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.136364 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istio-token\") pod \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\" (UID: \"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05\") " Jul 07 16:07:39.136648 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.136534 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3b882148-c0e8-4e0b-bc16-9aa88e6e7f05" (UID: "3b882148-c0e8-4e0b-bc16-9aa88e6e7f05"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:07:39.136749 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.136685 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3b882148-c0e8-4e0b-bc16-9aa88e6e7f05" (UID: "3b882148-c0e8-4e0b-bc16-9aa88e6e7f05"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:07:39.136749 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.136695 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-workload-socket\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:07:39.136842 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.136756 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3b882148-c0e8-4e0b-bc16-9aa88e6e7f05" (UID: "3b882148-c0e8-4e0b-bc16-9aa88e6e7f05"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:07:39.137020 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.136996 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istio-data" (OuterVolumeSpecName: "istio-data") pod "3b882148-c0e8-4e0b-bc16-9aa88e6e7f05" (UID: "3b882148-c0e8-4e0b-bc16-9aa88e6e7f05"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:07:39.137020 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.137009 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3b882148-c0e8-4e0b-bc16-9aa88e6e7f05" (UID: "3b882148-c0e8-4e0b-bc16-9aa88e6e7f05"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:07:39.139066 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.139035 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3b882148-c0e8-4e0b-bc16-9aa88e6e7f05" (UID: "3b882148-c0e8-4e0b-bc16-9aa88e6e7f05"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:07:39.139185 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.139068 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-kube-api-access-5kdv8" (OuterVolumeSpecName: "kube-api-access-5kdv8") pod "3b882148-c0e8-4e0b-bc16-9aa88e6e7f05" (UID: "3b882148-c0e8-4e0b-bc16-9aa88e6e7f05"). InnerVolumeSpecName "kube-api-access-5kdv8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:07:39.139185 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.139122 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istio-token" (OuterVolumeSpecName: "istio-token") pod "3b882148-c0e8-4e0b-bc16-9aa88e6e7f05" (UID: "3b882148-c0e8-4e0b-bc16-9aa88e6e7f05"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:07:39.139185 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.139124 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3b882148-c0e8-4e0b-bc16-9aa88e6e7f05" (UID: "3b882148-c0e8-4e0b-bc16-9aa88e6e7f05"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 16:07:39.238049 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.238023 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-credential-socket\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:07:39.238049 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.238047 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-workload-certs\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:07:39.238049 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.238056 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istiod-ca-cert\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:07:39.238270 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.238064 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istio-token\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:07:39.238270 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.238073 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istio-envoy\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:07:39.238270 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.238084 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kdv8\" (UniqueName: \"kubernetes.io/projected/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-kube-api-access-5kdv8\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:07:39.238270 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.238092 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istio-data\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:07:39.238270 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.238100 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05-istio-podinfo\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:07:39.505233 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.505197 2575 generic.go:356] "Generic (PLEG): container finished" podID="3b882148-c0e8-4e0b-bc16-9aa88e6e7f05" containerID="760ada191c13c06518e3dab916bbe80f8a53a1f4913941069ef62dfb9269ce54" exitCode=0 Jul 07 16:07:39.505378 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.505237 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" Jul 07 16:07:39.505378 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.505272 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" event={"ID":"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05","Type":"ContainerDied","Data":"760ada191c13c06518e3dab916bbe80f8a53a1f4913941069ef62dfb9269ce54"} Jul 07 16:07:39.505378 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.505327 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h" event={"ID":"3b882148-c0e8-4e0b-bc16-9aa88e6e7f05","Type":"ContainerDied","Data":"fe096f59e8c3506dea21a3a519a0c47d6f3737bb039f1eec604d967c5a5da53d"} Jul 07 16:07:39.505378 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.505358 2575 scope.go:117] "RemoveContainer" containerID="760ada191c13c06518e3dab916bbe80f8a53a1f4913941069ef62dfb9269ce54" Jul 07 16:07:39.514134 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.514116 2575 scope.go:117] "RemoveContainer" containerID="760ada191c13c06518e3dab916bbe80f8a53a1f4913941069ef62dfb9269ce54" Jul 07 16:07:39.514370 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:07:39.514352 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"760ada191c13c06518e3dab916bbe80f8a53a1f4913941069ef62dfb9269ce54\": container with ID starting with 760ada191c13c06518e3dab916bbe80f8a53a1f4913941069ef62dfb9269ce54 not found: ID does not exist" containerID="760ada191c13c06518e3dab916bbe80f8a53a1f4913941069ef62dfb9269ce54" Jul 07 16:07:39.514415 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.514378 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"760ada191c13c06518e3dab916bbe80f8a53a1f4913941069ef62dfb9269ce54"} err="failed to get container status \"760ada191c13c06518e3dab916bbe80f8a53a1f4913941069ef62dfb9269ce54\": rpc error: code = NotFound desc = could not find container \"760ada191c13c06518e3dab916bbe80f8a53a1f4913941069ef62dfb9269ce54\": container with ID starting with 760ada191c13c06518e3dab916bbe80f8a53a1f4913941069ef62dfb9269ce54 not found: ID does not exist" Jul 07 16:07:39.526445 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.526422 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h"] Jul 07 16:07:39.531516 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:39.531494 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b203da5c-openshift-default-6c687fb8c-bdp9h"] Jul 07 16:07:41.436170 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:41.436133 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b882148-c0e8-4e0b-bc16-9aa88e6e7f05" path="/var/lib/kubelet/pods/3b882148-c0e8-4e0b-bc16-9aa88e6e7f05/volumes" Jul 07 16:07:47.821459 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:47.821428 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h"] Jul 07 16:07:47.821823 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:47.821799 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b882148-c0e8-4e0b-bc16-9aa88e6e7f05" containerName="istio-proxy" Jul 07 16:07:47.821823 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:47.821809 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b882148-c0e8-4e0b-bc16-9aa88e6e7f05" containerName="istio-proxy" Jul 07 16:07:47.821823 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:47.821819 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f66871d-0b81-484d-9556-48b025f7cfe1" containerName="maas-api" Jul 07 16:07:47.821823 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:47.821824 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f66871d-0b81-484d-9556-48b025f7cfe1" containerName="maas-api" Jul 07 16:07:47.821996 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:47.821892 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b882148-c0e8-4e0b-bc16-9aa88e6e7f05" containerName="istio-proxy" Jul 07 16:07:47.821996 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:47.821904 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f66871d-0b81-484d-9556-48b025f7cfe1" containerName="maas-api" Jul 07 16:07:47.825116 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:47.825097 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h" Jul 07 16:07:47.830502 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:47.830312 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-f839a472-serving-cert\"" Jul 07 16:07:47.830502 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:47.830395 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-5vqnq\"" Jul 07 16:07:47.916235 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:47.916199 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e52334b6-f7cf-48ef-83a7-a672d26aa9d0-openshift-service-ca\") pod \"maas-api-e2e-derive-f839a472-544cd84d84-zkn4h\" (UID: \"e52334b6-f7cf-48ef-83a7-a672d26aa9d0\") " pod="opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h" Jul 07 16:07:47.916390 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:47.916291 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6d7g\" (UniqueName: \"kubernetes.io/projected/e52334b6-f7cf-48ef-83a7-a672d26aa9d0-kube-api-access-z6d7g\") pod \"maas-api-e2e-derive-f839a472-544cd84d84-zkn4h\" (UID: \"e52334b6-f7cf-48ef-83a7-a672d26aa9d0\") " pod="opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h" Jul 07 16:07:47.916390 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:47.916338 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e52334b6-f7cf-48ef-83a7-a672d26aa9d0-maas-api-tls\") pod \"maas-api-e2e-derive-f839a472-544cd84d84-zkn4h\" (UID: \"e52334b6-f7cf-48ef-83a7-a672d26aa9d0\") " pod="opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h" Jul 07 16:07:47.955032 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:47.954990 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h"] Jul 07 16:07:48.017433 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:48.017395 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6d7g\" (UniqueName: \"kubernetes.io/projected/e52334b6-f7cf-48ef-83a7-a672d26aa9d0-kube-api-access-z6d7g\") pod \"maas-api-e2e-derive-f839a472-544cd84d84-zkn4h\" (UID: \"e52334b6-f7cf-48ef-83a7-a672d26aa9d0\") " pod="opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h" Jul 07 16:07:48.017626 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:48.017451 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e52334b6-f7cf-48ef-83a7-a672d26aa9d0-maas-api-tls\") pod \"maas-api-e2e-derive-f839a472-544cd84d84-zkn4h\" (UID: \"e52334b6-f7cf-48ef-83a7-a672d26aa9d0\") " pod="opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h" Jul 07 16:07:48.017626 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:48.017492 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e52334b6-f7cf-48ef-83a7-a672d26aa9d0-openshift-service-ca\") pod \"maas-api-e2e-derive-f839a472-544cd84d84-zkn4h\" (UID: \"e52334b6-f7cf-48ef-83a7-a672d26aa9d0\") " pod="opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h" Jul 07 16:07:48.017763 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:07:48.017625 2575 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-f839a472-serving-cert: secret "maas-api-e2e-derive-f839a472-serving-cert" not found Jul 07 16:07:48.017763 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:07:48.017721 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e52334b6-f7cf-48ef-83a7-a672d26aa9d0-maas-api-tls podName:e52334b6-f7cf-48ef-83a7-a672d26aa9d0 nodeName:}" failed. No retries permitted until 2026-07-07 16:07:48.517697106 +0000 UTC m=+1555.612978176 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e52334b6-f7cf-48ef-83a7-a672d26aa9d0-maas-api-tls") pod "maas-api-e2e-derive-f839a472-544cd84d84-zkn4h" (UID: "e52334b6-f7cf-48ef-83a7-a672d26aa9d0") : secret "maas-api-e2e-derive-f839a472-serving-cert" not found Jul 07 16:07:48.018154 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:48.018134 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e52334b6-f7cf-48ef-83a7-a672d26aa9d0-openshift-service-ca\") pod \"maas-api-e2e-derive-f839a472-544cd84d84-zkn4h\" (UID: \"e52334b6-f7cf-48ef-83a7-a672d26aa9d0\") " pod="opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h" Jul 07 16:07:48.044146 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:48.044117 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6d7g\" (UniqueName: \"kubernetes.io/projected/e52334b6-f7cf-48ef-83a7-a672d26aa9d0-kube-api-access-z6d7g\") pod \"maas-api-e2e-derive-f839a472-544cd84d84-zkn4h\" (UID: \"e52334b6-f7cf-48ef-83a7-a672d26aa9d0\") " pod="opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h" Jul 07 16:07:48.523680 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:48.523637 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e52334b6-f7cf-48ef-83a7-a672d26aa9d0-maas-api-tls\") pod \"maas-api-e2e-derive-f839a472-544cd84d84-zkn4h\" (UID: \"e52334b6-f7cf-48ef-83a7-a672d26aa9d0\") " pod="opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h" Jul 07 16:07:48.526088 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:48.526063 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e52334b6-f7cf-48ef-83a7-a672d26aa9d0-maas-api-tls\") pod \"maas-api-e2e-derive-f839a472-544cd84d84-zkn4h\" (UID: \"e52334b6-f7cf-48ef-83a7-a672d26aa9d0\") " pod="opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h" Jul 07 16:07:48.735758 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:48.735723 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h" Jul 07 16:07:48.891907 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:48.891368 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h"] Jul 07 16:07:49.560259 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:49.560161 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h" event={"ID":"e52334b6-f7cf-48ef-83a7-a672d26aa9d0","Type":"ContainerStarted","Data":"51c51f77bba559ecb7d2cee65d65948525343219387f0a462ce72cc1459a4534"} Jul 07 16:07:49.560259 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:49.560223 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h" event={"ID":"e52334b6-f7cf-48ef-83a7-a672d26aa9d0","Type":"ContainerStarted","Data":"a117a7182a22888bb3799c8516fffef5226509fcef7d04c2a4048121c4b653ad"} Jul 07 16:07:49.560488 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:49.560261 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h" Jul 07 16:07:49.593584 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:49.593539 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h" podStartSLOduration=2.3330290639999998 podStartE2EDuration="2.593527036s" totalImagesPullingTime="260.497972ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:07:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:07:49.593355763 +0000 UTC m=+1556.688636841" watchObservedRunningTime="2026-07-07 16:07:49.593527036 +0000 UTC m=+1556.688808114" Jul 07 16:07:52.754745 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:52.754714 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h"] Jul 07 16:07:52.755213 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:52.754905 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h" podUID="e52334b6-f7cf-48ef-83a7-a672d26aa9d0" containerName="maas-api" containerID="cri-o://51c51f77bba559ecb7d2cee65d65948525343219387f0a462ce72cc1459a4534" gracePeriod=30 Jul 07 16:07:52.995388 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:52.995363 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h" Jul 07 16:07:53.062614 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:53.062529 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6d7g\" (UniqueName: \"kubernetes.io/projected/e52334b6-f7cf-48ef-83a7-a672d26aa9d0-kube-api-access-z6d7g\") pod \"e52334b6-f7cf-48ef-83a7-a672d26aa9d0\" (UID: \"e52334b6-f7cf-48ef-83a7-a672d26aa9d0\") " Jul 07 16:07:53.062761 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:53.062639 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e52334b6-f7cf-48ef-83a7-a672d26aa9d0-openshift-service-ca\") pod \"e52334b6-f7cf-48ef-83a7-a672d26aa9d0\" (UID: \"e52334b6-f7cf-48ef-83a7-a672d26aa9d0\") " Jul 07 16:07:53.062761 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:53.062666 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e52334b6-f7cf-48ef-83a7-a672d26aa9d0-maas-api-tls\") pod \"e52334b6-f7cf-48ef-83a7-a672d26aa9d0\" (UID: \"e52334b6-f7cf-48ef-83a7-a672d26aa9d0\") " Jul 07 16:07:53.063050 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:53.063019 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e52334b6-f7cf-48ef-83a7-a672d26aa9d0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e52334b6-f7cf-48ef-83a7-a672d26aa9d0" (UID: "e52334b6-f7cf-48ef-83a7-a672d26aa9d0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:07:53.064700 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:53.064673 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e52334b6-f7cf-48ef-83a7-a672d26aa9d0-kube-api-access-z6d7g" (OuterVolumeSpecName: "kube-api-access-z6d7g") pod "e52334b6-f7cf-48ef-83a7-a672d26aa9d0" (UID: "e52334b6-f7cf-48ef-83a7-a672d26aa9d0"). InnerVolumeSpecName "kube-api-access-z6d7g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:07:53.064782 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:53.064755 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e52334b6-f7cf-48ef-83a7-a672d26aa9d0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e52334b6-f7cf-48ef-83a7-a672d26aa9d0" (UID: "e52334b6-f7cf-48ef-83a7-a672d26aa9d0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:07:53.164106 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:53.164072 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e52334b6-f7cf-48ef-83a7-a672d26aa9d0-openshift-service-ca\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:07:53.164106 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:53.164100 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e52334b6-f7cf-48ef-83a7-a672d26aa9d0-maas-api-tls\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:07:53.164106 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:53.164109 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6d7g\" (UniqueName: \"kubernetes.io/projected/e52334b6-f7cf-48ef-83a7-a672d26aa9d0-kube-api-access-z6d7g\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:07:53.581472 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:53.581440 2575 generic.go:356] "Generic (PLEG): container finished" podID="e52334b6-f7cf-48ef-83a7-a672d26aa9d0" containerID="51c51f77bba559ecb7d2cee65d65948525343219387f0a462ce72cc1459a4534" exitCode=0 Jul 07 16:07:53.581841 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:53.581483 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h" Jul 07 16:07:53.581841 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:53.581523 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h" event={"ID":"e52334b6-f7cf-48ef-83a7-a672d26aa9d0","Type":"ContainerDied","Data":"51c51f77bba559ecb7d2cee65d65948525343219387f0a462ce72cc1459a4534"} Jul 07 16:07:53.581841 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:53.581579 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h" event={"ID":"e52334b6-f7cf-48ef-83a7-a672d26aa9d0","Type":"ContainerDied","Data":"a117a7182a22888bb3799c8516fffef5226509fcef7d04c2a4048121c4b653ad"} Jul 07 16:07:53.581841 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:53.581608 2575 scope.go:117] "RemoveContainer" containerID="51c51f77bba559ecb7d2cee65d65948525343219387f0a462ce72cc1459a4534" Jul 07 16:07:53.590019 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:53.589991 2575 scope.go:117] "RemoveContainer" containerID="51c51f77bba559ecb7d2cee65d65948525343219387f0a462ce72cc1459a4534" Jul 07 16:07:53.590275 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:07:53.590255 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51c51f77bba559ecb7d2cee65d65948525343219387f0a462ce72cc1459a4534\": container with ID starting with 51c51f77bba559ecb7d2cee65d65948525343219387f0a462ce72cc1459a4534 not found: ID does not exist" containerID="51c51f77bba559ecb7d2cee65d65948525343219387f0a462ce72cc1459a4534" Jul 07 16:07:53.590325 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:53.590286 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51c51f77bba559ecb7d2cee65d65948525343219387f0a462ce72cc1459a4534"} err="failed to get container status \"51c51f77bba559ecb7d2cee65d65948525343219387f0a462ce72cc1459a4534\": rpc error: code = NotFound desc = could not find container \"51c51f77bba559ecb7d2cee65d65948525343219387f0a462ce72cc1459a4534\": container with ID starting with 51c51f77bba559ecb7d2cee65d65948525343219387f0a462ce72cc1459a4534 not found: ID does not exist" Jul 07 16:07:53.606529 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:53.606495 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h"] Jul 07 16:07:53.617701 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:53.617677 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-f839a472-544cd84d84-zkn4h"] Jul 07 16:07:55.435833 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:07:55.435796 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e52334b6-f7cf-48ef-83a7-a672d26aa9d0" path="/var/lib/kubelet/pods/e52334b6-f7cf-48ef-83a7-a672d26aa9d0/volumes" Jul 07 16:09:42.188250 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.188219 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s"] Jul 07 16:09:42.188721 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.188586 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e52334b6-f7cf-48ef-83a7-a672d26aa9d0" containerName="maas-api" Jul 07 16:09:42.188721 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.188597 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e52334b6-f7cf-48ef-83a7-a672d26aa9d0" containerName="maas-api" Jul 07 16:09:42.188721 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.188681 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e52334b6-f7cf-48ef-83a7-a672d26aa9d0" containerName="maas-api" Jul 07 16:09:42.192079 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.192058 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.194678 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.194662 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-acfc1ab1-openshift-default-dockercfg-pt2j6\"" Jul 07 16:09:42.220904 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.220879 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s"] Jul 07 16:09:42.245835 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.245799 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fb25fa4-5808-4301-acd3-240647b0e7f3-istiod-ca-cert\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.246028 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.245903 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-workload-socket\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.246028 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.245928 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-istio-data\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.246028 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.245968 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-istio-envoy\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.246028 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.246021 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-workload-certs\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.246252 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.246072 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fb25fa4-5808-4301-acd3-240647b0e7f3-istio-podinfo\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.246252 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.246099 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fb25fa4-5808-4301-acd3-240647b0e7f3-istio-token\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.246252 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.246145 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-credential-socket\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.246252 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.246203 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz4tp\" (UniqueName: \"kubernetes.io/projected/5fb25fa4-5808-4301-acd3-240647b0e7f3-kube-api-access-vz4tp\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.346890 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.346832 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fb25fa4-5808-4301-acd3-240647b0e7f3-istiod-ca-cert\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.347087 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.346918 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-workload-socket\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.347087 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.346941 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-istio-data\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.347087 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.346981 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-istio-envoy\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.347087 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.347016 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-workload-certs\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.347087 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.347069 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fb25fa4-5808-4301-acd3-240647b0e7f3-istio-podinfo\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.347348 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.347093 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fb25fa4-5808-4301-acd3-240647b0e7f3-istio-token\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.347348 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.347129 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-credential-socket\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.347348 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.347184 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz4tp\" (UniqueName: \"kubernetes.io/projected/5fb25fa4-5808-4301-acd3-240647b0e7f3-kube-api-access-vz4tp\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.347490 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.347353 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-workload-socket\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.347490 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.347403 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-istio-data\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.347490 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.347460 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-workload-certs\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.347658 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.347637 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fb25fa4-5808-4301-acd3-240647b0e7f3-istiod-ca-cert\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.347706 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.347641 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-credential-socket\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.349503 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.349476 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-istio-envoy\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.349614 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.349585 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fb25fa4-5808-4301-acd3-240647b0e7f3-istio-podinfo\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.357106 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.357037 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fb25fa4-5808-4301-acd3-240647b0e7f3-istio-token\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.357487 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.357456 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz4tp\" (UniqueName: \"kubernetes.io/projected/5fb25fa4-5808-4301-acd3-240647b0e7f3-kube-api-access-vz4tp\") pod \"e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.502893 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.502781 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:42.638008 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.637981 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s"] Jul 07 16:09:42.640047 ip-10-0-131-221 kubenswrapper[2575]: W0707 16:09:42.640020 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fb25fa4_5808_4301_acd3_240647b0e7f3.slice/crio-2216ef39c179b2db8ed964eded3e1dd2dfc0056709f32361aa1b13874894739c WatchSource:0}: Error finding container 2216ef39c179b2db8ed964eded3e1dd2dfc0056709f32361aa1b13874894739c: Status 404 returned error can't find the container with id 2216ef39c179b2db8ed964eded3e1dd2dfc0056709f32361aa1b13874894739c Jul 07 16:09:42.642615 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.642576 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:09:42.642709 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.642639 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:09:42.642709 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:42.642668 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:09:43.091823 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:43.091784 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" event={"ID":"5fb25fa4-5808-4301-acd3-240647b0e7f3","Type":"ContainerStarted","Data":"5b155c9e802375d8d74abe758b7761b413c92a485444c7567aeeca5c1f9e34d7"} Jul 07 16:09:43.092025 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:43.091840 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" event={"ID":"5fb25fa4-5808-4301-acd3-240647b0e7f3","Type":"ContainerStarted","Data":"2216ef39c179b2db8ed964eded3e1dd2dfc0056709f32361aa1b13874894739c"} Jul 07 16:09:43.123298 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:43.123239 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" podStartSLOduration=1.123223718 podStartE2EDuration="1.123223718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:09:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:09:43.118408959 +0000 UTC m=+1670.213690038" watchObservedRunningTime="2026-07-07 16:09:43.123223718 +0000 UTC m=+1670.218504838" Jul 07 16:09:43.503675 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:43.503575 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:43.508357 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:43.508332 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:44.096206 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:44.096171 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:44.097182 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:44.097153 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:09:45.208306 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:45.208270 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n"] Jul 07 16:09:45.211740 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:45.211715 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n" Jul 07 16:09:45.214585 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:45.214564 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-acfc1ab1-serving-cert\"" Jul 07 16:09:45.214675 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:45.214653 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-5vqnq\"" Jul 07 16:09:45.225039 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:45.225017 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n"] Jul 07 16:09:45.276742 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:45.276711 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de6898a9-3417-4d87-acd5-822a67c60c47-maas-api-tls\") pod \"maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n\" (UID: \"de6898a9-3417-4d87-acd5-822a67c60c47\") " pod="opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n" Jul 07 16:09:45.276891 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:45.276748 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de6898a9-3417-4d87-acd5-822a67c60c47-openshift-service-ca\") pod \"maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n\" (UID: \"de6898a9-3417-4d87-acd5-822a67c60c47\") " pod="opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n" Jul 07 16:09:45.276961 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:45.276895 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx4z2\" (UniqueName: \"kubernetes.io/projected/de6898a9-3417-4d87-acd5-822a67c60c47-kube-api-access-nx4z2\") pod \"maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n\" (UID: \"de6898a9-3417-4d87-acd5-822a67c60c47\") " pod="opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n" Jul 07 16:09:45.378115 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:45.378078 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx4z2\" (UniqueName: \"kubernetes.io/projected/de6898a9-3417-4d87-acd5-822a67c60c47-kube-api-access-nx4z2\") pod \"maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n\" (UID: \"de6898a9-3417-4d87-acd5-822a67c60c47\") " pod="opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n" Jul 07 16:09:45.378319 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:45.378165 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de6898a9-3417-4d87-acd5-822a67c60c47-maas-api-tls\") pod \"maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n\" (UID: \"de6898a9-3417-4d87-acd5-822a67c60c47\") " pod="opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n" Jul 07 16:09:45.378319 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:45.378193 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de6898a9-3417-4d87-acd5-822a67c60c47-openshift-service-ca\") pod \"maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n\" (UID: \"de6898a9-3417-4d87-acd5-822a67c60c47\") " pod="opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n" Jul 07 16:09:45.378319 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:09:45.378252 2575 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-acfc1ab1-serving-cert: secret "maas-api-e2e-mt-acfc1ab1-serving-cert" not found Jul 07 16:09:45.378492 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:09:45.378334 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de6898a9-3417-4d87-acd5-822a67c60c47-maas-api-tls podName:de6898a9-3417-4d87-acd5-822a67c60c47 nodeName:}" failed. No retries permitted until 2026-07-07 16:09:45.878315999 +0000 UTC m=+1672.973597069 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/de6898a9-3417-4d87-acd5-822a67c60c47-maas-api-tls") pod "maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n" (UID: "de6898a9-3417-4d87-acd5-822a67c60c47") : secret "maas-api-e2e-mt-acfc1ab1-serving-cert" not found Jul 07 16:09:45.378802 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:45.378774 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de6898a9-3417-4d87-acd5-822a67c60c47-openshift-service-ca\") pod \"maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n\" (UID: \"de6898a9-3417-4d87-acd5-822a67c60c47\") " pod="opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n" Jul 07 16:09:45.392648 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:45.392626 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx4z2\" (UniqueName: \"kubernetes.io/projected/de6898a9-3417-4d87-acd5-822a67c60c47-kube-api-access-nx4z2\") pod \"maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n\" (UID: \"de6898a9-3417-4d87-acd5-822a67c60c47\") " pod="opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n" Jul 07 16:09:45.882621 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:45.882584 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de6898a9-3417-4d87-acd5-822a67c60c47-maas-api-tls\") pod \"maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n\" (UID: \"de6898a9-3417-4d87-acd5-822a67c60c47\") " pod="opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n" Jul 07 16:09:45.885066 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:45.885040 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de6898a9-3417-4d87-acd5-822a67c60c47-maas-api-tls\") pod \"maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n\" (UID: \"de6898a9-3417-4d87-acd5-822a67c60c47\") " pod="opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n" Jul 07 16:09:46.122332 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:46.122299 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n" Jul 07 16:09:46.255993 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:46.255966 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n"] Jul 07 16:09:46.257966 ip-10-0-131-221 kubenswrapper[2575]: W0707 16:09:46.257937 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde6898a9_3417_4d87_acd5_822a67c60c47.slice/crio-e1e00f9ae1243473f3c667c3b1df810cc1aaf8d8c4c888956cdbd98996afc56b WatchSource:0}: Error finding container e1e00f9ae1243473f3c667c3b1df810cc1aaf8d8c4c888956cdbd98996afc56b: Status 404 returned error can't find the container with id e1e00f9ae1243473f3c667c3b1df810cc1aaf8d8c4c888956cdbd98996afc56b Jul 07 16:09:47.111512 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:47.111479 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n" event={"ID":"de6898a9-3417-4d87-acd5-822a67c60c47","Type":"ContainerStarted","Data":"1ffea57437f8ad4e02dc4852579df6a3495a114aed3e43a9c778dfbadada0436"} Jul 07 16:09:47.111744 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:47.111530 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n" event={"ID":"de6898a9-3417-4d87-acd5-822a67c60c47","Type":"ContainerStarted","Data":"e1e00f9ae1243473f3c667c3b1df810cc1aaf8d8c4c888956cdbd98996afc56b"} Jul 07 16:09:47.111744 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:47.111622 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n" Jul 07 16:09:47.141993 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:47.141943 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n" podStartSLOduration=1.691065271 podStartE2EDuration="2.141928367s" totalImagesPullingTime="450.863096ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:09:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:09:47.138743899 +0000 UTC m=+1674.234024990" watchObservedRunningTime="2026-07-07 16:09:47.141928367 +0000 UTC m=+1674.237209445" Jul 07 16:09:53.122683 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:53.122651 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n" Jul 07 16:09:58.131011 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:58.130973 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n"] Jul 07 16:09:58.131480 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:58.131252 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n" podUID="de6898a9-3417-4d87-acd5-822a67c60c47" containerName="maas-api" containerID="cri-o://1ffea57437f8ad4e02dc4852579df6a3495a114aed3e43a9c778dfbadada0436" gracePeriod=30 Jul 07 16:09:58.380741 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:58.380716 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n" Jul 07 16:09:58.515060 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:58.514981 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de6898a9-3417-4d87-acd5-822a67c60c47-openshift-service-ca\") pod \"de6898a9-3417-4d87-acd5-822a67c60c47\" (UID: \"de6898a9-3417-4d87-acd5-822a67c60c47\") " Jul 07 16:09:58.515060 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:58.515014 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nx4z2\" (UniqueName: \"kubernetes.io/projected/de6898a9-3417-4d87-acd5-822a67c60c47-kube-api-access-nx4z2\") pod \"de6898a9-3417-4d87-acd5-822a67c60c47\" (UID: \"de6898a9-3417-4d87-acd5-822a67c60c47\") " Jul 07 16:09:58.515293 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:58.515124 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de6898a9-3417-4d87-acd5-822a67c60c47-maas-api-tls\") pod \"de6898a9-3417-4d87-acd5-822a67c60c47\" (UID: \"de6898a9-3417-4d87-acd5-822a67c60c47\") " Jul 07 16:09:58.515376 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:58.515345 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de6898a9-3417-4d87-acd5-822a67c60c47-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "de6898a9-3417-4d87-acd5-822a67c60c47" (UID: "de6898a9-3417-4d87-acd5-822a67c60c47"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:09:58.517240 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:58.517206 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de6898a9-3417-4d87-acd5-822a67c60c47-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "de6898a9-3417-4d87-acd5-822a67c60c47" (UID: "de6898a9-3417-4d87-acd5-822a67c60c47"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:09:58.517240 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:58.517218 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de6898a9-3417-4d87-acd5-822a67c60c47-kube-api-access-nx4z2" (OuterVolumeSpecName: "kube-api-access-nx4z2") pod "de6898a9-3417-4d87-acd5-822a67c60c47" (UID: "de6898a9-3417-4d87-acd5-822a67c60c47"). InnerVolumeSpecName "kube-api-access-nx4z2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:09:58.616398 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:58.616369 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de6898a9-3417-4d87-acd5-822a67c60c47-maas-api-tls\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:09:58.616398 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:58.616398 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de6898a9-3417-4d87-acd5-822a67c60c47-openshift-service-ca\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:09:58.616398 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:58.616408 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nx4z2\" (UniqueName: \"kubernetes.io/projected/de6898a9-3417-4d87-acd5-822a67c60c47-kube-api-access-nx4z2\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:09:59.098222 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:59.098183 2575 patch_prober.go:28] interesting pod/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 07 16:09:59.098451 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:59.098244 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" podUID="5fb25fa4-5808-4301-acd3-240647b0e7f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 07 16:09:59.176405 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:59.176378 2575 generic.go:356] "Generic (PLEG): container finished" podID="5fb25fa4-5808-4301-acd3-240647b0e7f3" containerID="5b155c9e802375d8d74abe758b7761b413c92a485444c7567aeeca5c1f9e34d7" exitCode=0 Jul 07 16:09:59.176801 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:59.176465 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" event={"ID":"5fb25fa4-5808-4301-acd3-240647b0e7f3","Type":"ContainerDied","Data":"5b155c9e802375d8d74abe758b7761b413c92a485444c7567aeeca5c1f9e34d7"} Jul 07 16:09:59.176947 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:59.176923 2575 scope.go:117] "RemoveContainer" containerID="5b155c9e802375d8d74abe758b7761b413c92a485444c7567aeeca5c1f9e34d7" Jul 07 16:09:59.177775 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:59.177757 2575 generic.go:356] "Generic (PLEG): container finished" podID="de6898a9-3417-4d87-acd5-822a67c60c47" containerID="1ffea57437f8ad4e02dc4852579df6a3495a114aed3e43a9c778dfbadada0436" exitCode=0 Jul 07 16:09:59.177840 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:59.177809 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n" event={"ID":"de6898a9-3417-4d87-acd5-822a67c60c47","Type":"ContainerDied","Data":"1ffea57437f8ad4e02dc4852579df6a3495a114aed3e43a9c778dfbadada0436"} Jul 07 16:09:59.177906 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:59.177839 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n" event={"ID":"de6898a9-3417-4d87-acd5-822a67c60c47","Type":"ContainerDied","Data":"e1e00f9ae1243473f3c667c3b1df810cc1aaf8d8c4c888956cdbd98996afc56b"} Jul 07 16:09:59.177906 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:59.177857 2575 scope.go:117] "RemoveContainer" containerID="1ffea57437f8ad4e02dc4852579df6a3495a114aed3e43a9c778dfbadada0436" Jul 07 16:09:59.177993 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:59.177928 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n" Jul 07 16:09:59.178042 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:59.177991 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:09:59.178095 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:59.178072 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:09:59.178152 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:59.178121 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:09:59.190540 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:59.190517 2575 scope.go:117] "RemoveContainer" containerID="1ffea57437f8ad4e02dc4852579df6a3495a114aed3e43a9c778dfbadada0436" Jul 07 16:09:59.190824 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:09:59.190801 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ffea57437f8ad4e02dc4852579df6a3495a114aed3e43a9c778dfbadada0436\": container with ID starting with 1ffea57437f8ad4e02dc4852579df6a3495a114aed3e43a9c778dfbadada0436 not found: ID does not exist" containerID="1ffea57437f8ad4e02dc4852579df6a3495a114aed3e43a9c778dfbadada0436" Jul 07 16:09:59.190949 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:59.190829 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ffea57437f8ad4e02dc4852579df6a3495a114aed3e43a9c778dfbadada0436"} err="failed to get container status \"1ffea57437f8ad4e02dc4852579df6a3495a114aed3e43a9c778dfbadada0436\": rpc error: code = NotFound desc = could not find container \"1ffea57437f8ad4e02dc4852579df6a3495a114aed3e43a9c778dfbadada0436\": container with ID starting with 1ffea57437f8ad4e02dc4852579df6a3495a114aed3e43a9c778dfbadada0436 not found: ID does not exist" Jul 07 16:09:59.219821 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:59.219791 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n"] Jul 07 16:09:59.225064 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:59.225039 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-acfc1ab1-85cbd68d56-64b4n"] Jul 07 16:09:59.437021 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:09:59.436947 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de6898a9-3417-4d87-acd5-822a67c60c47" path="/var/lib/kubelet/pods/de6898a9-3417-4d87-acd5-822a67c60c47/volumes" Jul 07 16:10:00.185321 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:00.185287 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" event={"ID":"5fb25fa4-5808-4301-acd3-240647b0e7f3","Type":"ContainerStarted","Data":"e3cb529be18a92a890e834b9ffb94e3f1cafe285072e714a9b53b5fde4556e05"} Jul 07 16:10:00.503299 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:00.503195 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:10:00.503474 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:00.503450 2575 patch_prober.go:28] interesting pod/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 07 16:10:00.503529 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:00.503495 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" podUID="5fb25fa4-5808-4301-acd3-240647b0e7f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 07 16:10:01.504228 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:01.504187 2575 patch_prober.go:28] interesting pod/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 07 16:10:01.504626 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:01.504247 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" podUID="5fb25fa4-5808-4301-acd3-240647b0e7f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 07 16:10:02.503724 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:02.503680 2575 patch_prober.go:28] interesting pod/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 07 16:10:02.503979 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:02.503742 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" podUID="5fb25fa4-5808-4301-acd3-240647b0e7f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 07 16:10:03.503775 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:03.503734 2575 patch_prober.go:28] interesting pod/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 07 16:10:03.504300 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:03.503814 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" podUID="5fb25fa4-5808-4301-acd3-240647b0e7f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 07 16:10:04.507963 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:04.507933 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:10:04.508348 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:04.508207 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:10:04.508841 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:04.508826 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:10:05.386669 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:05.386635 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s"] Jul 07 16:10:06.213964 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:06.213932 2575 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" secret="" err="secret \"e2e-mt-acfc1ab1-openshift-default-dockercfg-pt2j6\" not found" Jul 07 16:10:07.218358 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:07.218320 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" podUID="5fb25fa4-5808-4301-acd3-240647b0e7f3" containerName="istio-proxy" containerID="cri-o://e3cb529be18a92a890e834b9ffb94e3f1cafe285072e714a9b53b5fde4556e05" gracePeriod=30 Jul 07 16:10:12.478334 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.478311 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:10:12.557652 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.557567 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-workload-socket\") pod \"5fb25fa4-5808-4301-acd3-240647b0e7f3\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " Jul 07 16:10:12.557652 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.557611 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-istio-data\") pod \"5fb25fa4-5808-4301-acd3-240647b0e7f3\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " Jul 07 16:10:12.557899 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.557666 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fb25fa4-5808-4301-acd3-240647b0e7f3-istio-podinfo\") pod \"5fb25fa4-5808-4301-acd3-240647b0e7f3\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " Jul 07 16:10:12.557899 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.557690 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-credential-socket\") pod \"5fb25fa4-5808-4301-acd3-240647b0e7f3\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " Jul 07 16:10:12.557899 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.557720 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-istio-envoy\") pod \"5fb25fa4-5808-4301-acd3-240647b0e7f3\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " Jul 07 16:10:12.557899 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.557751 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fb25fa4-5808-4301-acd3-240647b0e7f3-istio-token\") pod \"5fb25fa4-5808-4301-acd3-240647b0e7f3\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " Jul 07 16:10:12.557899 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.557819 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fb25fa4-5808-4301-acd3-240647b0e7f3-istiod-ca-cert\") pod \"5fb25fa4-5808-4301-acd3-240647b0e7f3\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " Jul 07 16:10:12.558233 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.557904 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-workload-certs\") pod \"5fb25fa4-5808-4301-acd3-240647b0e7f3\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " Jul 07 16:10:12.558233 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.557913 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5fb25fa4-5808-4301-acd3-240647b0e7f3" (UID: "5fb25fa4-5808-4301-acd3-240647b0e7f3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:10:12.558233 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.557929 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vz4tp\" (UniqueName: \"kubernetes.io/projected/5fb25fa4-5808-4301-acd3-240647b0e7f3-kube-api-access-vz4tp\") pod \"5fb25fa4-5808-4301-acd3-240647b0e7f3\" (UID: \"5fb25fa4-5808-4301-acd3-240647b0e7f3\") " Jul 07 16:10:12.558233 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.558030 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5fb25fa4-5808-4301-acd3-240647b0e7f3" (UID: "5fb25fa4-5808-4301-acd3-240647b0e7f3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:10:12.558233 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.558131 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5fb25fa4-5808-4301-acd3-240647b0e7f3" (UID: "5fb25fa4-5808-4301-acd3-240647b0e7f3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:10:12.558491 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.558353 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fb25fa4-5808-4301-acd3-240647b0e7f3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5fb25fa4-5808-4301-acd3-240647b0e7f3" (UID: "5fb25fa4-5808-4301-acd3-240647b0e7f3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:10:12.558491 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.558367 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-workload-certs\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:10:12.558491 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.558386 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-workload-socket\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:10:12.558491 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.558400 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-credential-socket\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:10:12.558709 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.558647 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-istio-data" (OuterVolumeSpecName: "istio-data") pod "5fb25fa4-5808-4301-acd3-240647b0e7f3" (UID: "5fb25fa4-5808-4301-acd3-240647b0e7f3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:10:12.560146 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.560115 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5fb25fa4-5808-4301-acd3-240647b0e7f3" (UID: "5fb25fa4-5808-4301-acd3-240647b0e7f3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:10:12.560248 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.560154 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fb25fa4-5808-4301-acd3-240647b0e7f3-kube-api-access-vz4tp" (OuterVolumeSpecName: "kube-api-access-vz4tp") pod "5fb25fa4-5808-4301-acd3-240647b0e7f3" (UID: "5fb25fa4-5808-4301-acd3-240647b0e7f3"). InnerVolumeSpecName "kube-api-access-vz4tp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:10:12.560392 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.560368 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5fb25fa4-5808-4301-acd3-240647b0e7f3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5fb25fa4-5808-4301-acd3-240647b0e7f3" (UID: "5fb25fa4-5808-4301-acd3-240647b0e7f3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 16:10:12.560489 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.560448 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fb25fa4-5808-4301-acd3-240647b0e7f3-istio-token" (OuterVolumeSpecName: "istio-token") pod "5fb25fa4-5808-4301-acd3-240647b0e7f3" (UID: "5fb25fa4-5808-4301-acd3-240647b0e7f3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:10:12.659415 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.659385 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-istio-data\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:10:12.659415 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.659410 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fb25fa4-5808-4301-acd3-240647b0e7f3-istio-podinfo\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:10:12.659415 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.659419 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fb25fa4-5808-4301-acd3-240647b0e7f3-istio-envoy\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:10:12.659637 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.659428 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fb25fa4-5808-4301-acd3-240647b0e7f3-istio-token\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:10:12.659637 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.659436 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fb25fa4-5808-4301-acd3-240647b0e7f3-istiod-ca-cert\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:10:12.659637 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:12.659444 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vz4tp\" (UniqueName: \"kubernetes.io/projected/5fb25fa4-5808-4301-acd3-240647b0e7f3-kube-api-access-vz4tp\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:10:13.249240 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:13.249200 2575 generic.go:356] "Generic (PLEG): container finished" podID="5fb25fa4-5808-4301-acd3-240647b0e7f3" containerID="e3cb529be18a92a890e834b9ffb94e3f1cafe285072e714a9b53b5fde4556e05" exitCode=0 Jul 07 16:10:13.249418 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:13.249251 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" Jul 07 16:10:13.249418 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:13.249290 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" event={"ID":"5fb25fa4-5808-4301-acd3-240647b0e7f3","Type":"ContainerDied","Data":"e3cb529be18a92a890e834b9ffb94e3f1cafe285072e714a9b53b5fde4556e05"} Jul 07 16:10:13.249418 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:13.249350 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s" event={"ID":"5fb25fa4-5808-4301-acd3-240647b0e7f3","Type":"ContainerDied","Data":"2216ef39c179b2db8ed964eded3e1dd2dfc0056709f32361aa1b13874894739c"} Jul 07 16:10:13.249418 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:13.249375 2575 scope.go:117] "RemoveContainer" containerID="e3cb529be18a92a890e834b9ffb94e3f1cafe285072e714a9b53b5fde4556e05" Jul 07 16:10:13.258702 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:13.258684 2575 scope.go:117] "RemoveContainer" containerID="5b155c9e802375d8d74abe758b7761b413c92a485444c7567aeeca5c1f9e34d7" Jul 07 16:10:13.266887 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:13.266855 2575 scope.go:117] "RemoveContainer" containerID="e3cb529be18a92a890e834b9ffb94e3f1cafe285072e714a9b53b5fde4556e05" Jul 07 16:10:13.267152 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:10:13.267133 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3cb529be18a92a890e834b9ffb94e3f1cafe285072e714a9b53b5fde4556e05\": container with ID starting with e3cb529be18a92a890e834b9ffb94e3f1cafe285072e714a9b53b5fde4556e05 not found: ID does not exist" containerID="e3cb529be18a92a890e834b9ffb94e3f1cafe285072e714a9b53b5fde4556e05" Jul 07 16:10:13.267231 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:13.267157 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3cb529be18a92a890e834b9ffb94e3f1cafe285072e714a9b53b5fde4556e05"} err="failed to get container status \"e3cb529be18a92a890e834b9ffb94e3f1cafe285072e714a9b53b5fde4556e05\": rpc error: code = NotFound desc = could not find container \"e3cb529be18a92a890e834b9ffb94e3f1cafe285072e714a9b53b5fde4556e05\": container with ID starting with e3cb529be18a92a890e834b9ffb94e3f1cafe285072e714a9b53b5fde4556e05 not found: ID does not exist" Jul 07 16:10:13.267231 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:13.267175 2575 scope.go:117] "RemoveContainer" containerID="5b155c9e802375d8d74abe758b7761b413c92a485444c7567aeeca5c1f9e34d7" Jul 07 16:10:13.267471 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:10:13.267454 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b155c9e802375d8d74abe758b7761b413c92a485444c7567aeeca5c1f9e34d7\": container with ID starting with 5b155c9e802375d8d74abe758b7761b413c92a485444c7567aeeca5c1f9e34d7 not found: ID does not exist" containerID="5b155c9e802375d8d74abe758b7761b413c92a485444c7567aeeca5c1f9e34d7" Jul 07 16:10:13.267520 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:13.267473 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b155c9e802375d8d74abe758b7761b413c92a485444c7567aeeca5c1f9e34d7"} err="failed to get container status \"5b155c9e802375d8d74abe758b7761b413c92a485444c7567aeeca5c1f9e34d7\": rpc error: code = NotFound desc = could not find container \"5b155c9e802375d8d74abe758b7761b413c92a485444c7567aeeca5c1f9e34d7\": container with ID starting with 5b155c9e802375d8d74abe758b7761b413c92a485444c7567aeeca5c1f9e34d7 not found: ID does not exist" Jul 07 16:10:13.274126 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:13.274102 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s"] Jul 07 16:10:13.280420 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:13.280395 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-acfc1ab1-openshift-default-5d5865ffff-rrs4s"] Jul 07 16:10:13.437192 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:13.437158 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fb25fa4-5808-4301-acd3-240647b0e7f3" path="/var/lib/kubelet/pods/5fb25fa4-5808-4301-acd3-240647b0e7f3/volumes" Jul 07 16:10:26.934192 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:26.934114 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm"] Jul 07 16:10:26.934573 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:26.934553 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fb25fa4-5808-4301-acd3-240647b0e7f3" containerName="istio-proxy" Jul 07 16:10:26.934617 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:26.934576 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fb25fa4-5808-4301-acd3-240647b0e7f3" containerName="istio-proxy" Jul 07 16:10:26.934617 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:26.934589 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fb25fa4-5808-4301-acd3-240647b0e7f3" containerName="istio-proxy" Jul 07 16:10:26.934617 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:26.934595 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fb25fa4-5808-4301-acd3-240647b0e7f3" containerName="istio-proxy" Jul 07 16:10:26.934617 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:26.934608 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de6898a9-3417-4d87-acd5-822a67c60c47" containerName="maas-api" Jul 07 16:10:26.934617 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:26.934613 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="de6898a9-3417-4d87-acd5-822a67c60c47" containerName="maas-api" Jul 07 16:10:26.934778 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:26.934682 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="de6898a9-3417-4d87-acd5-822a67c60c47" containerName="maas-api" Jul 07 16:10:26.934778 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:26.934689 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fb25fa4-5808-4301-acd3-240647b0e7f3" containerName="istio-proxy" Jul 07 16:10:26.934778 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:26.934696 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fb25fa4-5808-4301-acd3-240647b0e7f3" containerName="istio-proxy" Jul 07 16:10:26.937407 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:26.937386 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:26.941973 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:26.941951 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2f31ab17-openshift-default-dockercfg-rtndj\"" Jul 07 16:10:26.964050 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:26.964023 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm"] Jul 07 16:10:27.093089 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.093055 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istio-podinfo\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.093285 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.093120 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istiod-ca-cert\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.093285 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.093208 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbvc9\" (UniqueName: \"kubernetes.io/projected/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-kube-api-access-wbvc9\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.093285 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.093246 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-workload-certs\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.093419 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.093301 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istio-token\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.093419 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.093350 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-credential-socket\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.093419 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.093384 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istio-data\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.093419 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.093410 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-workload-socket\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.093590 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.093452 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istio-envoy\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.194934 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.194820 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-workload-socket\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.194934 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.194877 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istio-envoy\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.194934 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.194910 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istio-podinfo\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.195226 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.194973 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istiod-ca-cert\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.195226 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.195008 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbvc9\" (UniqueName: \"kubernetes.io/projected/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-kube-api-access-wbvc9\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.195226 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.195033 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-workload-certs\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.195226 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.195090 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istio-token\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.195226 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.195128 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-credential-socket\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.195226 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.195167 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istio-data\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.195584 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.195262 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-workload-socket\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.195584 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.195446 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-workload-certs\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.195584 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.195518 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istio-data\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.195701 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.195616 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-credential-socket\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.195737 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.195711 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istiod-ca-cert\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.197250 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.197225 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istio-envoy\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.197426 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.197411 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istio-podinfo\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.206341 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.206320 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istio-token\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.209111 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.209088 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbvc9\" (UniqueName: \"kubernetes.io/projected/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-kube-api-access-wbvc9\") pod \"e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.246901 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.246859 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:27.398426 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.398392 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm"] Jul 07 16:10:27.401554 ip-10-0-131-221 kubenswrapper[2575]: W0707 16:10:27.401526 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac2c0e80_21b7_4f8e_b445_dbc8c38e4710.slice/crio-f03a9c02eec88d39dfedf487b679389577811c3aeab422498e37926a63ada01b WatchSource:0}: Error finding container f03a9c02eec88d39dfedf487b679389577811c3aeab422498e37926a63ada01b: Status 404 returned error can't find the container with id f03a9c02eec88d39dfedf487b679389577811c3aeab422498e37926a63ada01b Jul 07 16:10:27.403626 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.403594 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:10:27.403701 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.403666 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:10:27.403701 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:27.403697 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:10:28.322313 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:28.322271 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" event={"ID":"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710","Type":"ContainerStarted","Data":"6b1189dee10fc7dc12a993a1a94aa4f8139e9c9e66d09a723976ce5e424f8461"} Jul 07 16:10:28.322313 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:28.322320 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" event={"ID":"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710","Type":"ContainerStarted","Data":"f03a9c02eec88d39dfedf487b679389577811c3aeab422498e37926a63ada01b"} Jul 07 16:10:28.349208 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:28.349157 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" podStartSLOduration=2.3491417549999998 podStartE2EDuration="2.349141755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:10:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:10:28.347386532 +0000 UTC m=+1715.442667612" watchObservedRunningTime="2026-07-07 16:10:28.349141755 +0000 UTC m=+1715.444422833" Jul 07 16:10:29.247215 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:29.247183 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:29.252182 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:29.252155 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:29.327312 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:29.327280 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:29.328242 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:29.328221 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:30.564527 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:30.564491 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9"] Jul 07 16:10:30.567750 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:30.567726 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9" Jul 07 16:10:30.570577 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:30.570555 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-5vqnq\"" Jul 07 16:10:30.570681 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:30.570602 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-2f31ab17-serving-cert\"" Jul 07 16:10:30.583791 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:30.583765 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9"] Jul 07 16:10:30.727268 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:30.727234 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1898db7f-b577-47ed-9cf3-53b6024bebc6-openshift-service-ca\") pod \"maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9\" (UID: \"1898db7f-b577-47ed-9cf3-53b6024bebc6\") " pod="opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9" Jul 07 16:10:30.727481 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:30.727284 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1898db7f-b577-47ed-9cf3-53b6024bebc6-maas-api-tls\") pod \"maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9\" (UID: \"1898db7f-b577-47ed-9cf3-53b6024bebc6\") " pod="opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9" Jul 07 16:10:30.727481 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:30.727378 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkrb4\" (UniqueName: \"kubernetes.io/projected/1898db7f-b577-47ed-9cf3-53b6024bebc6-kube-api-access-xkrb4\") pod \"maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9\" (UID: \"1898db7f-b577-47ed-9cf3-53b6024bebc6\") " pod="opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9" Jul 07 16:10:30.828784 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:30.828692 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1898db7f-b577-47ed-9cf3-53b6024bebc6-openshift-service-ca\") pod \"maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9\" (UID: \"1898db7f-b577-47ed-9cf3-53b6024bebc6\") " pod="opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9" Jul 07 16:10:30.828784 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:30.828750 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1898db7f-b577-47ed-9cf3-53b6024bebc6-maas-api-tls\") pod \"maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9\" (UID: \"1898db7f-b577-47ed-9cf3-53b6024bebc6\") " pod="opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9" Jul 07 16:10:30.828784 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:30.828785 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkrb4\" (UniqueName: \"kubernetes.io/projected/1898db7f-b577-47ed-9cf3-53b6024bebc6-kube-api-access-xkrb4\") pod \"maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9\" (UID: \"1898db7f-b577-47ed-9cf3-53b6024bebc6\") " pod="opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9" Jul 07 16:10:30.829362 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:30.829338 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1898db7f-b577-47ed-9cf3-53b6024bebc6-openshift-service-ca\") pod \"maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9\" (UID: \"1898db7f-b577-47ed-9cf3-53b6024bebc6\") " pod="opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9" Jul 07 16:10:30.831148 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:30.831125 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1898db7f-b577-47ed-9cf3-53b6024bebc6-maas-api-tls\") pod \"maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9\" (UID: \"1898db7f-b577-47ed-9cf3-53b6024bebc6\") " pod="opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9" Jul 07 16:10:30.839460 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:30.839439 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkrb4\" (UniqueName: \"kubernetes.io/projected/1898db7f-b577-47ed-9cf3-53b6024bebc6-kube-api-access-xkrb4\") pod \"maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9\" (UID: \"1898db7f-b577-47ed-9cf3-53b6024bebc6\") " pod="opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9" Jul 07 16:10:30.886397 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:30.886368 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9" Jul 07 16:10:31.021044 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:31.021020 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9"] Jul 07 16:10:31.023079 ip-10-0-131-221 kubenswrapper[2575]: W0707 16:10:31.023038 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1898db7f_b577_47ed_9cf3_53b6024bebc6.slice/crio-20eddf68065babe254b8f0830ed16aacd67fba69992da9b560ab4ff8bea855d9 WatchSource:0}: Error finding container 20eddf68065babe254b8f0830ed16aacd67fba69992da9b560ab4ff8bea855d9: Status 404 returned error can't find the container with id 20eddf68065babe254b8f0830ed16aacd67fba69992da9b560ab4ff8bea855d9 Jul 07 16:10:31.339370 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:31.339332 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9" event={"ID":"1898db7f-b577-47ed-9cf3-53b6024bebc6","Type":"ContainerStarted","Data":"20eddf68065babe254b8f0830ed16aacd67fba69992da9b560ab4ff8bea855d9"} Jul 07 16:10:32.346023 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:32.345984 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9" event={"ID":"1898db7f-b577-47ed-9cf3-53b6024bebc6","Type":"ContainerStarted","Data":"d31fe1025041aaa97ed1e70c1e410fde1db0a2074df2315d1b09fb6360cc8b0f"} Jul 07 16:10:32.346533 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:32.346096 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9" Jul 07 16:10:32.369365 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:32.369320 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9" podStartSLOduration=1.842099121 podStartE2EDuration="2.369302675s" totalImagesPullingTime="527.203554ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:10:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:10:32.366696073 +0000 UTC m=+1719.461977152" watchObservedRunningTime="2026-07-07 16:10:32.369302675 +0000 UTC m=+1719.464583754" Jul 07 16:10:38.370238 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:38.370209 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9" Jul 07 16:10:43.403228 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:43.403195 2575 generic.go:356] "Generic (PLEG): container finished" podID="ac2c0e80-21b7-4f8e-b445-dbc8c38e4710" containerID="6b1189dee10fc7dc12a993a1a94aa4f8139e9c9e66d09a723976ce5e424f8461" exitCode=0 Jul 07 16:10:43.403688 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:43.403272 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" event={"ID":"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710","Type":"ContainerDied","Data":"6b1189dee10fc7dc12a993a1a94aa4f8139e9c9e66d09a723976ce5e424f8461"} Jul 07 16:10:43.403754 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:43.403730 2575 scope.go:117] "RemoveContainer" containerID="6b1189dee10fc7dc12a993a1a94aa4f8139e9c9e66d09a723976ce5e424f8461" Jul 07 16:10:43.404819 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:43.404789 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:10:43.404900 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:43.404857 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:10:43.404946 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:43.404901 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:10:44.328211 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:44.328168 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:44.410231 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:44.410195 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" event={"ID":"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710","Type":"ContainerStarted","Data":"fdf156794016696f471abe57bcbdab67a75a08335bc972883e35dcdc9faaf516"} Jul 07 16:10:45.248063 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:45.248024 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:45.249574 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:45.249543 2575 patch_prober.go:28] interesting pod/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 07 16:10:45.249791 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:45.249766 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" podUID="ac2c0e80-21b7-4f8e-b445-dbc8c38e4710" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 07 16:10:46.248297 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:46.248259 2575 patch_prober.go:28] interesting pod/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 07 16:10:46.248930 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:46.248896 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" podUID="ac2c0e80-21b7-4f8e-b445-dbc8c38e4710" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 07 16:10:47.247585 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:47.247541 2575 patch_prober.go:28] interesting pod/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 07 16:10:47.247806 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:47.247606 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" podUID="ac2c0e80-21b7-4f8e-b445-dbc8c38e4710" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 07 16:10:48.247429 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:48.247382 2575 patch_prober.go:28] interesting pod/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 07 16:10:48.247934 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:48.247449 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" podUID="ac2c0e80-21b7-4f8e-b445-dbc8c38e4710" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 07 16:10:49.251965 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:49.251936 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:49.252412 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:49.252363 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:10:49.253059 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:10:49.253041 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:11:12.954027 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:12.953966 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9"] Jul 07 16:11:12.954676 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:12.954242 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9" podUID="1898db7f-b577-47ed-9cf3-53b6024bebc6" containerName="maas-api" containerID="cri-o://d31fe1025041aaa97ed1e70c1e410fde1db0a2074df2315d1b09fb6360cc8b0f" gracePeriod=30 Jul 07 16:11:13.202652 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:13.202622 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9" Jul 07 16:11:13.341700 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:13.341671 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1898db7f-b577-47ed-9cf3-53b6024bebc6-openshift-service-ca\") pod \"1898db7f-b577-47ed-9cf3-53b6024bebc6\" (UID: \"1898db7f-b577-47ed-9cf3-53b6024bebc6\") " Jul 07 16:11:13.341889 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:13.341769 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1898db7f-b577-47ed-9cf3-53b6024bebc6-maas-api-tls\") pod \"1898db7f-b577-47ed-9cf3-53b6024bebc6\" (UID: \"1898db7f-b577-47ed-9cf3-53b6024bebc6\") " Jul 07 16:11:13.341889 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:13.341844 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkrb4\" (UniqueName: \"kubernetes.io/projected/1898db7f-b577-47ed-9cf3-53b6024bebc6-kube-api-access-xkrb4\") pod \"1898db7f-b577-47ed-9cf3-53b6024bebc6\" (UID: \"1898db7f-b577-47ed-9cf3-53b6024bebc6\") " Jul 07 16:11:13.342177 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:13.342139 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1898db7f-b577-47ed-9cf3-53b6024bebc6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1898db7f-b577-47ed-9cf3-53b6024bebc6" (UID: "1898db7f-b577-47ed-9cf3-53b6024bebc6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:11:13.343963 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:13.343940 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1898db7f-b577-47ed-9cf3-53b6024bebc6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1898db7f-b577-47ed-9cf3-53b6024bebc6" (UID: "1898db7f-b577-47ed-9cf3-53b6024bebc6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:11:13.343963 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:13.343950 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1898db7f-b577-47ed-9cf3-53b6024bebc6-kube-api-access-xkrb4" (OuterVolumeSpecName: "kube-api-access-xkrb4") pod "1898db7f-b577-47ed-9cf3-53b6024bebc6" (UID: "1898db7f-b577-47ed-9cf3-53b6024bebc6"). InnerVolumeSpecName "kube-api-access-xkrb4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:11:13.443026 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:13.442857 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1898db7f-b577-47ed-9cf3-53b6024bebc6-maas-api-tls\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:11:13.443026 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:13.442904 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkrb4\" (UniqueName: \"kubernetes.io/projected/1898db7f-b577-47ed-9cf3-53b6024bebc6-kube-api-access-xkrb4\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:11:13.443026 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:13.442914 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1898db7f-b577-47ed-9cf3-53b6024bebc6-openshift-service-ca\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:11:13.550479 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:13.550445 2575 generic.go:356] "Generic (PLEG): container finished" podID="1898db7f-b577-47ed-9cf3-53b6024bebc6" containerID="d31fe1025041aaa97ed1e70c1e410fde1db0a2074df2315d1b09fb6360cc8b0f" exitCode=0 Jul 07 16:11:13.550633 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:13.550490 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9" Jul 07 16:11:13.550633 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:13.550505 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9" event={"ID":"1898db7f-b577-47ed-9cf3-53b6024bebc6","Type":"ContainerDied","Data":"d31fe1025041aaa97ed1e70c1e410fde1db0a2074df2315d1b09fb6360cc8b0f"} Jul 07 16:11:13.550633 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:13.550551 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9" event={"ID":"1898db7f-b577-47ed-9cf3-53b6024bebc6","Type":"ContainerDied","Data":"20eddf68065babe254b8f0830ed16aacd67fba69992da9b560ab4ff8bea855d9"} Jul 07 16:11:13.550633 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:13.550574 2575 scope.go:117] "RemoveContainer" containerID="d31fe1025041aaa97ed1e70c1e410fde1db0a2074df2315d1b09fb6360cc8b0f" Jul 07 16:11:13.559488 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:13.559463 2575 scope.go:117] "RemoveContainer" containerID="d31fe1025041aaa97ed1e70c1e410fde1db0a2074df2315d1b09fb6360cc8b0f" Jul 07 16:11:13.559795 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:11:13.559772 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d31fe1025041aaa97ed1e70c1e410fde1db0a2074df2315d1b09fb6360cc8b0f\": container with ID starting with d31fe1025041aaa97ed1e70c1e410fde1db0a2074df2315d1b09fb6360cc8b0f not found: ID does not exist" containerID="d31fe1025041aaa97ed1e70c1e410fde1db0a2074df2315d1b09fb6360cc8b0f" Jul 07 16:11:13.559846 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:13.559811 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d31fe1025041aaa97ed1e70c1e410fde1db0a2074df2315d1b09fb6360cc8b0f"} err="failed to get container status \"d31fe1025041aaa97ed1e70c1e410fde1db0a2074df2315d1b09fb6360cc8b0f\": rpc error: code = NotFound desc = could not find container \"d31fe1025041aaa97ed1e70c1e410fde1db0a2074df2315d1b09fb6360cc8b0f\": container with ID starting with d31fe1025041aaa97ed1e70c1e410fde1db0a2074df2315d1b09fb6360cc8b0f not found: ID does not exist" Jul 07 16:11:13.569723 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:13.569693 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9"] Jul 07 16:11:13.575604 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:13.575577 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-2f31ab17-7b7c657f9c-kfdm9"] Jul 07 16:11:15.436084 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:15.436052 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1898db7f-b577-47ed-9cf3-53b6024bebc6" path="/var/lib/kubelet/pods/1898db7f-b577-47ed-9cf3-53b6024bebc6/volumes" Jul 07 16:11:53.474470 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:53.474442 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nfh8b_162f95df-a98f-49a6-a91e-689a8e22be46/console-operator/1.log" Jul 07 16:11:53.475264 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:53.475244 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nfh8b_162f95df-a98f-49a6-a91e-689a8e22be46/console-operator/1.log" Jul 07 16:11:53.480986 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:53.480966 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4d86s_fbf78c29-89f3-4aad-82a7-2e8f99b6356e/ovn-acl-logging/0.log" Jul 07 16:11:53.481566 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:11:53.481545 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4d86s_fbf78c29-89f3-4aad-82a7-2e8f99b6356e/ovn-acl-logging/0.log" Jul 07 16:12:08.915513 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:08.915478 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm"] Jul 07 16:12:08.915990 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:08.915714 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" podUID="ac2c0e80-21b7-4f8e-b445-dbc8c38e4710" containerName="istio-proxy" containerID="cri-o://fdf156794016696f471abe57bcbdab67a75a08335bc972883e35dcdc9faaf516" gracePeriod=30 Jul 07 16:12:14.172653 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.172625 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:12:14.290899 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.290773 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbvc9\" (UniqueName: \"kubernetes.io/projected/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-kube-api-access-wbvc9\") pod \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " Jul 07 16:12:14.290899 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.290892 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-workload-certs\") pod \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " Jul 07 16:12:14.291123 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.290934 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istio-token\") pod \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " Jul 07 16:12:14.291123 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.290955 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-workload-socket\") pod \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " Jul 07 16:12:14.291123 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.290981 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istiod-ca-cert\") pod \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " Jul 07 16:12:14.291123 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.291020 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istio-podinfo\") pod \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " Jul 07 16:12:14.291123 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.291045 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-credential-socket\") pod \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " Jul 07 16:12:14.291123 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.291079 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istio-envoy\") pod \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " Jul 07 16:12:14.291123 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.291123 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istio-data\") pod \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\" (UID: \"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710\") " Jul 07 16:12:14.291486 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.291258 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ac2c0e80-21b7-4f8e-b445-dbc8c38e4710" (UID: "ac2c0e80-21b7-4f8e-b445-dbc8c38e4710"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:12:14.291486 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.291273 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ac2c0e80-21b7-4f8e-b445-dbc8c38e4710" (UID: "ac2c0e80-21b7-4f8e-b445-dbc8c38e4710"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:12:14.291486 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.291465 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-workload-certs\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:12:14.291486 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.291484 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-workload-socket\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:12:14.291671 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.291488 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ac2c0e80-21b7-4f8e-b445-dbc8c38e4710" (UID: "ac2c0e80-21b7-4f8e-b445-dbc8c38e4710"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:12:14.291811 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.291787 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ac2c0e80-21b7-4f8e-b445-dbc8c38e4710" (UID: "ac2c0e80-21b7-4f8e-b445-dbc8c38e4710"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:12:14.292231 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.292207 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istio-data" (OuterVolumeSpecName: "istio-data") pod "ac2c0e80-21b7-4f8e-b445-dbc8c38e4710" (UID: "ac2c0e80-21b7-4f8e-b445-dbc8c38e4710"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:12:14.293096 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.293075 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-kube-api-access-wbvc9" (OuterVolumeSpecName: "kube-api-access-wbvc9") pod "ac2c0e80-21b7-4f8e-b445-dbc8c38e4710" (UID: "ac2c0e80-21b7-4f8e-b445-dbc8c38e4710"). InnerVolumeSpecName "kube-api-access-wbvc9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:12:14.293544 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.293518 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ac2c0e80-21b7-4f8e-b445-dbc8c38e4710" (UID: "ac2c0e80-21b7-4f8e-b445-dbc8c38e4710"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:12:14.293670 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.293542 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ac2c0e80-21b7-4f8e-b445-dbc8c38e4710" (UID: "ac2c0e80-21b7-4f8e-b445-dbc8c38e4710"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 16:12:14.293670 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.293548 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istio-token" (OuterVolumeSpecName: "istio-token") pod "ac2c0e80-21b7-4f8e-b445-dbc8c38e4710" (UID: "ac2c0e80-21b7-4f8e-b445-dbc8c38e4710"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:12:14.392875 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.392833 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbvc9\" (UniqueName: \"kubernetes.io/projected/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-kube-api-access-wbvc9\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:12:14.393052 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.392898 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istio-token\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:12:14.393052 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.392908 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istiod-ca-cert\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:12:14.393052 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.392916 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istio-podinfo\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:12:14.393052 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.392924 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-credential-socket\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:12:14.393052 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.392933 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istio-envoy\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:12:14.393052 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.392941 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710-istio-data\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:12:14.842941 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.842902 2575 generic.go:356] "Generic (PLEG): container finished" podID="ac2c0e80-21b7-4f8e-b445-dbc8c38e4710" containerID="fdf156794016696f471abe57bcbdab67a75a08335bc972883e35dcdc9faaf516" exitCode=0 Jul 07 16:12:14.843124 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.842963 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" Jul 07 16:12:14.843124 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.842957 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" event={"ID":"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710","Type":"ContainerDied","Data":"fdf156794016696f471abe57bcbdab67a75a08335bc972883e35dcdc9faaf516"} Jul 07 16:12:14.843124 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.843016 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm" event={"ID":"ac2c0e80-21b7-4f8e-b445-dbc8c38e4710","Type":"ContainerDied","Data":"f03a9c02eec88d39dfedf487b679389577811c3aeab422498e37926a63ada01b"} Jul 07 16:12:14.843124 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.843038 2575 scope.go:117] "RemoveContainer" containerID="fdf156794016696f471abe57bcbdab67a75a08335bc972883e35dcdc9faaf516" Jul 07 16:12:14.852246 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.852224 2575 scope.go:117] "RemoveContainer" containerID="6b1189dee10fc7dc12a993a1a94aa4f8139e9c9e66d09a723976ce5e424f8461" Jul 07 16:12:14.859843 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.859827 2575 scope.go:117] "RemoveContainer" containerID="fdf156794016696f471abe57bcbdab67a75a08335bc972883e35dcdc9faaf516" Jul 07 16:12:14.860114 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:12:14.860093 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdf156794016696f471abe57bcbdab67a75a08335bc972883e35dcdc9faaf516\": container with ID starting with fdf156794016696f471abe57bcbdab67a75a08335bc972883e35dcdc9faaf516 not found: ID does not exist" containerID="fdf156794016696f471abe57bcbdab67a75a08335bc972883e35dcdc9faaf516" Jul 07 16:12:14.860173 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.860129 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdf156794016696f471abe57bcbdab67a75a08335bc972883e35dcdc9faaf516"} err="failed to get container status \"fdf156794016696f471abe57bcbdab67a75a08335bc972883e35dcdc9faaf516\": rpc error: code = NotFound desc = could not find container \"fdf156794016696f471abe57bcbdab67a75a08335bc972883e35dcdc9faaf516\": container with ID starting with fdf156794016696f471abe57bcbdab67a75a08335bc972883e35dcdc9faaf516 not found: ID does not exist" Jul 07 16:12:14.860173 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.860148 2575 scope.go:117] "RemoveContainer" containerID="6b1189dee10fc7dc12a993a1a94aa4f8139e9c9e66d09a723976ce5e424f8461" Jul 07 16:12:14.860444 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:12:14.860421 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b1189dee10fc7dc12a993a1a94aa4f8139e9c9e66d09a723976ce5e424f8461\": container with ID starting with 6b1189dee10fc7dc12a993a1a94aa4f8139e9c9e66d09a723976ce5e424f8461 not found: ID does not exist" containerID="6b1189dee10fc7dc12a993a1a94aa4f8139e9c9e66d09a723976ce5e424f8461" Jul 07 16:12:14.860536 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.860448 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b1189dee10fc7dc12a993a1a94aa4f8139e9c9e66d09a723976ce5e424f8461"} err="failed to get container status \"6b1189dee10fc7dc12a993a1a94aa4f8139e9c9e66d09a723976ce5e424f8461\": rpc error: code = NotFound desc = could not find container \"6b1189dee10fc7dc12a993a1a94aa4f8139e9c9e66d09a723976ce5e424f8461\": container with ID starting with 6b1189dee10fc7dc12a993a1a94aa4f8139e9c9e66d09a723976ce5e424f8461 not found: ID does not exist" Jul 07 16:12:14.866854 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.866832 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm"] Jul 07 16:12:14.872732 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:14.872711 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2f31ab17-openshift-default-85cd895587-6vtqm"] Jul 07 16:12:15.436639 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:12:15.436607 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac2c0e80-21b7-4f8e-b445-dbc8c38e4710" path="/var/lib/kubelet/pods/ac2c0e80-21b7-4f8e-b445-dbc8c38e4710/volumes" Jul 07 16:13:06.396644 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.396602 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc"] Jul 07 16:13:06.397070 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.396990 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1898db7f-b577-47ed-9cf3-53b6024bebc6" containerName="maas-api" Jul 07 16:13:06.397070 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.397003 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1898db7f-b577-47ed-9cf3-53b6024bebc6" containerName="maas-api" Jul 07 16:13:06.397070 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.397013 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac2c0e80-21b7-4f8e-b445-dbc8c38e4710" containerName="istio-proxy" Jul 07 16:13:06.397070 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.397018 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac2c0e80-21b7-4f8e-b445-dbc8c38e4710" containerName="istio-proxy" Jul 07 16:13:06.397207 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.397086 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1898db7f-b577-47ed-9cf3-53b6024bebc6" containerName="maas-api" Jul 07 16:13:06.397207 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.397096 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac2c0e80-21b7-4f8e-b445-dbc8c38e4710" containerName="istio-proxy" Jul 07 16:13:06.397207 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.397103 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac2c0e80-21b7-4f8e-b445-dbc8c38e4710" containerName="istio-proxy" Jul 07 16:13:06.399153 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.399136 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.401879 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.401843 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-d4320a-openshift-default-dockercfg-7b7gm\"" Jul 07 16:13:06.415043 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.415014 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc"] Jul 07 16:13:06.479590 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.479559 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istio-envoy\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.479590 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.479594 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-credential-socket\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.479795 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.479698 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istiod-ca-cert\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.479795 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.479724 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istio-podinfo\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.479795 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.479745 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istio-data\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.479795 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.479767 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istio-token\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.479948 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.479877 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w275v\" (UniqueName: \"kubernetes.io/projected/d5708e9b-da63-4c0e-81ef-67b35bf784e1-kube-api-access-w275v\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.479948 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.479926 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-workload-socket\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.480018 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.479952 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-workload-certs\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.580524 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.580496 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istiod-ca-cert\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.580524 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.580528 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istio-podinfo\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.580782 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.580550 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istio-data\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.580782 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.580566 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istio-token\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.580782 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.580634 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w275v\" (UniqueName: \"kubernetes.io/projected/d5708e9b-da63-4c0e-81ef-67b35bf784e1-kube-api-access-w275v\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.580782 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.580662 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-workload-socket\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.580782 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.580753 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-workload-certs\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.581093 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.580817 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istio-envoy\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.581093 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.580847 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-credential-socket\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.581093 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.580988 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istio-data\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.581262 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.581169 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-workload-certs\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.581262 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.581245 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-credential-socket\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.581341 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.581264 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-workload-socket\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.581430 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.581411 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istiod-ca-cert\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.583497 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.583473 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istio-envoy\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.583725 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.583702 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istio-podinfo\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.592090 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.592025 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istio-token\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.592090 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.592025 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w275v\" (UniqueName: \"kubernetes.io/projected/d5708e9b-da63-4c0e-81ef-67b35bf784e1-kube-api-access-w275v\") pod \"e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.709040 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.708938 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:06.848082 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.848057 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc"] Jul 07 16:13:06.849622 ip-10-0-131-221 kubenswrapper[2575]: W0707 16:13:06.849595 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5708e9b_da63_4c0e_81ef_67b35bf784e1.slice/crio-36acf58c4d93e0ed3c7bf931db89c5efd81613902b5f8f24f2e0d2cf94613ac0 WatchSource:0}: Error finding container 36acf58c4d93e0ed3c7bf931db89c5efd81613902b5f8f24f2e0d2cf94613ac0: Status 404 returned error can't find the container with id 36acf58c4d93e0ed3c7bf931db89c5efd81613902b5f8f24f2e0d2cf94613ac0 Jul 07 16:13:06.851790 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.851758 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:13:06.851932 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.851830 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:13:06.851932 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:06.851896 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:13:07.092526 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:07.092491 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" event={"ID":"d5708e9b-da63-4c0e-81ef-67b35bf784e1","Type":"ContainerStarted","Data":"be4bf1cd614755b3e89f4843d92af4e87f0301e005a8d8b3660407f06d4c31af"} Jul 07 16:13:07.092734 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:07.092550 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" event={"ID":"d5708e9b-da63-4c0e-81ef-67b35bf784e1","Type":"ContainerStarted","Data":"36acf58c4d93e0ed3c7bf931db89c5efd81613902b5f8f24f2e0d2cf94613ac0"} Jul 07 16:13:07.118340 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:07.118284 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" podStartSLOduration=1.118264401 podStartE2EDuration="1.118264401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:13:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:13:07.115636743 +0000 UTC m=+1874.210917824" watchObservedRunningTime="2026-07-07 16:13:07.118264401 +0000 UTC m=+1874.213545479" Jul 07 16:13:07.709723 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:07.709681 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:07.715028 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:07.715001 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:08.097782 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:08.097751 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:08.098681 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:08.098663 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:09.348050 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:09.348016 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf"] Jul 07 16:13:09.348641 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:09.348621 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac2c0e80-21b7-4f8e-b445-dbc8c38e4710" containerName="istio-proxy" Jul 07 16:13:09.348735 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:09.348644 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac2c0e80-21b7-4f8e-b445-dbc8c38e4710" containerName="istio-proxy" Jul 07 16:13:09.350771 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:09.350751 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf" Jul 07 16:13:09.355075 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:09.355053 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-d4320a-serving-cert\"" Jul 07 16:13:09.355193 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:09.355124 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-5vqnq\"" Jul 07 16:13:09.367459 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:09.367433 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf"] Jul 07 16:13:09.512043 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:09.512008 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bwbh\" (UniqueName: \"kubernetes.io/projected/3b742bd3-0e73-4d07-b92a-85b6a33f5443-kube-api-access-4bwbh\") pod \"maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf\" (UID: \"3b742bd3-0e73-4d07-b92a-85b6a33f5443\") " pod="opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf" Jul 07 16:13:09.512214 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:09.512110 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b742bd3-0e73-4d07-b92a-85b6a33f5443-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf\" (UID: \"3b742bd3-0e73-4d07-b92a-85b6a33f5443\") " pod="opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf" Jul 07 16:13:09.512214 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:09.512144 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b742bd3-0e73-4d07-b92a-85b6a33f5443-maas-api-tls\") pod \"maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf\" (UID: \"3b742bd3-0e73-4d07-b92a-85b6a33f5443\") " pod="opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf" Jul 07 16:13:09.613263 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:09.613178 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bwbh\" (UniqueName: \"kubernetes.io/projected/3b742bd3-0e73-4d07-b92a-85b6a33f5443-kube-api-access-4bwbh\") pod \"maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf\" (UID: \"3b742bd3-0e73-4d07-b92a-85b6a33f5443\") " pod="opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf" Jul 07 16:13:09.613440 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:09.613271 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b742bd3-0e73-4d07-b92a-85b6a33f5443-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf\" (UID: \"3b742bd3-0e73-4d07-b92a-85b6a33f5443\") " pod="opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf" Jul 07 16:13:09.613440 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:09.613317 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b742bd3-0e73-4d07-b92a-85b6a33f5443-maas-api-tls\") pod \"maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf\" (UID: \"3b742bd3-0e73-4d07-b92a-85b6a33f5443\") " pod="opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf" Jul 07 16:13:09.613966 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:09.613942 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b742bd3-0e73-4d07-b92a-85b6a33f5443-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf\" (UID: \"3b742bd3-0e73-4d07-b92a-85b6a33f5443\") " pod="opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf" Jul 07 16:13:09.615732 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:09.615714 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b742bd3-0e73-4d07-b92a-85b6a33f5443-maas-api-tls\") pod \"maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf\" (UID: \"3b742bd3-0e73-4d07-b92a-85b6a33f5443\") " pod="opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf" Jul 07 16:13:09.623710 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:09.623691 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bwbh\" (UniqueName: \"kubernetes.io/projected/3b742bd3-0e73-4d07-b92a-85b6a33f5443-kube-api-access-4bwbh\") pod \"maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf\" (UID: \"3b742bd3-0e73-4d07-b92a-85b6a33f5443\") " pod="opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf" Jul 07 16:13:09.661636 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:09.661615 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf" Jul 07 16:13:09.810005 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:09.809954 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf"] Jul 07 16:13:09.818680 ip-10-0-131-221 kubenswrapper[2575]: W0707 16:13:09.818651 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b742bd3_0e73_4d07_b92a_85b6a33f5443.slice/crio-4a5499c06f43452353cf6675f703ec9ec63c545f8b104287f144bc972fa59e48 WatchSource:0}: Error finding container 4a5499c06f43452353cf6675f703ec9ec63c545f8b104287f144bc972fa59e48: Status 404 returned error can't find the container with id 4a5499c06f43452353cf6675f703ec9ec63c545f8b104287f144bc972fa59e48 Jul 07 16:13:09.820250 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:09.820231 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 16:13:10.107961 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:10.107922 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf" event={"ID":"3b742bd3-0e73-4d07-b92a-85b6a33f5443","Type":"ContainerStarted","Data":"4a5499c06f43452353cf6675f703ec9ec63c545f8b104287f144bc972fa59e48"} Jul 07 16:13:11.114774 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:11.114728 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf" event={"ID":"3b742bd3-0e73-4d07-b92a-85b6a33f5443","Type":"ContainerStarted","Data":"b4dc3f5cfdfcf5103a8fc35cb7de5e63090438c17d614397edd1d6d7d4bacd6f"} Jul 07 16:13:11.115151 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:11.114922 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf" Jul 07 16:13:11.140122 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:11.140074 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf" podStartSLOduration=1.608853543 podStartE2EDuration="2.140060775s" totalImagesPullingTime="531.207232ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:13:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:13:11.138204372 +0000 UTC m=+1878.233485450" watchObservedRunningTime="2026-07-07 16:13:11.140060775 +0000 UTC m=+1878.235341853" Jul 07 16:13:14.279657 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.279626 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6"] Jul 07 16:13:14.282457 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.282426 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.285504 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.285480 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-eb74cf-openshift-default-dockercfg-cd2px\"" Jul 07 16:13:14.304542 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.304511 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6"] Jul 07 16:13:14.356726 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.356688 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istio-envoy\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.356920 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.356743 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-workload-certs\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.356920 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.356766 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istio-data\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.356920 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.356786 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istio-podinfo\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.356920 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.356808 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istiod-ca-cert\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.356920 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.356837 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-workload-socket\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.356920 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.356895 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-credential-socket\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.357197 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.356927 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istio-token\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.357197 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.356966 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfspj\" (UniqueName: \"kubernetes.io/projected/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-kube-api-access-xfspj\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.458114 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.458072 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istio-envoy\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.458323 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.458141 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-workload-certs\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.458323 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.458166 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istio-data\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.458323 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.458190 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istio-podinfo\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.458323 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.458214 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istiod-ca-cert\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.458323 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.458251 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-workload-socket\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.458567 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.458394 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-credential-socket\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.458567 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.458453 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istio-token\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.458567 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.458505 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfspj\" (UniqueName: \"kubernetes.io/projected/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-kube-api-access-xfspj\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.458733 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.458638 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-workload-certs\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.458733 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.458662 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-workload-socket\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.458836 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.458746 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-credential-socket\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.459045 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.459026 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istio-data\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.459109 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.459085 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istiod-ca-cert\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.460536 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.460517 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istio-envoy\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.460754 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.460731 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istio-podinfo\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.468081 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.468056 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istio-token\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.468244 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.468224 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfspj\" (UniqueName: \"kubernetes.io/projected/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-kube-api-access-xfspj\") pod \"e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.593550 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.593519 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:14.742958 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.742916 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6"] Jul 07 16:13:14.744114 ip-10-0-131-221 kubenswrapper[2575]: W0707 16:13:14.744078 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f22bc65_3cff_4b9a_8ba9_52899e65a55a.slice/crio-d26b1238b3a3b17a90105a5784003741c69e1bb7a2c6d5115dc8c2e247e20183 WatchSource:0}: Error finding container d26b1238b3a3b17a90105a5784003741c69e1bb7a2c6d5115dc8c2e247e20183: Status 404 returned error can't find the container with id d26b1238b3a3b17a90105a5784003741c69e1bb7a2c6d5115dc8c2e247e20183 Jul 07 16:13:14.746329 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.746290 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:13:14.746384 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.746361 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:13:14.746429 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:14.746387 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:13:15.136300 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:15.136259 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" event={"ID":"9f22bc65-3cff-4b9a-8ba9-52899e65a55a","Type":"ContainerStarted","Data":"04d666326b8113764f5413ccf1c10b73dd72a64d179bcec5e9650c81ea468368"} Jul 07 16:13:15.136487 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:15.136310 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" event={"ID":"9f22bc65-3cff-4b9a-8ba9-52899e65a55a","Type":"ContainerStarted","Data":"d26b1238b3a3b17a90105a5784003741c69e1bb7a2c6d5115dc8c2e247e20183"} Jul 07 16:13:15.168622 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:15.168574 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" podStartSLOduration=1.168558998 podStartE2EDuration="1.168558998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:13:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:13:15.166618055 +0000 UTC m=+1882.261899134" watchObservedRunningTime="2026-07-07 16:13:15.168558998 +0000 UTC m=+1882.263840077" Jul 07 16:13:15.594561 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:15.594521 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:15.599545 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:15.599520 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:16.141190 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:16.141154 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:16.142133 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:16.142110 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:17.125530 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:17.125491 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf" Jul 07 16:13:24.063722 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:24.062153 2575 patch_prober.go:28] interesting pod/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": read tcp 10.132.0.2:43738->10.132.0.40:15021: read: connection reset by peer" start-of-body= Jul 07 16:13:24.063722 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:24.062223 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" podUID="d5708e9b-da63-4c0e-81ef-67b35bf784e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": read tcp 10.132.0.2:43738->10.132.0.40:15021: read: connection reset by peer" Jul 07 16:13:24.189234 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:24.189199 2575 generic.go:356] "Generic (PLEG): container finished" podID="d5708e9b-da63-4c0e-81ef-67b35bf784e1" containerID="be4bf1cd614755b3e89f4843d92af4e87f0301e005a8d8b3660407f06d4c31af" exitCode=0 Jul 07 16:13:24.189406 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:24.189270 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" event={"ID":"d5708e9b-da63-4c0e-81ef-67b35bf784e1","Type":"ContainerDied","Data":"be4bf1cd614755b3e89f4843d92af4e87f0301e005a8d8b3660407f06d4c31af"} Jul 07 16:13:24.189684 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:24.189669 2575 scope.go:117] "RemoveContainer" containerID="be4bf1cd614755b3e89f4843d92af4e87f0301e005a8d8b3660407f06d4c31af" Jul 07 16:13:24.190798 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:24.190769 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:13:24.190918 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:24.190899 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:13:24.190963 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:24.190948 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:13:25.196236 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:25.196201 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" event={"ID":"d5708e9b-da63-4c0e-81ef-67b35bf784e1","Type":"ContainerStarted","Data":"5b18d801af0255295c6ab770900611a4ba57b146740152c887227496f3b25996"} Jul 07 16:13:25.710387 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:25.710290 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:25.710387 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:25.710333 2575 patch_prober.go:28] interesting pod/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 07 16:13:25.710387 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:25.710378 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" podUID="d5708e9b-da63-4c0e-81ef-67b35bf784e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 07 16:13:26.709324 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:26.709280 2575 patch_prober.go:28] interesting pod/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 07 16:13:26.709732 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:26.709340 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" podUID="d5708e9b-da63-4c0e-81ef-67b35bf784e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 07 16:13:27.709431 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:27.709386 2575 patch_prober.go:28] interesting pod/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 07 16:13:27.709897 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:27.709461 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" podUID="d5708e9b-da63-4c0e-81ef-67b35bf784e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 07 16:13:28.710063 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:28.710024 2575 patch_prober.go:28] interesting pod/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 07 16:13:28.710444 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:28.710078 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" podUID="d5708e9b-da63-4c0e-81ef-67b35bf784e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 07 16:13:29.712948 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:29.712911 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:29.713398 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:29.713379 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:29.714039 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:29.714016 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:13:31.248913 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:31.248397 2575 patch_prober.go:28] interesting pod/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": read tcp 10.132.0.2:33626->10.132.0.42:15021: read: connection reset by peer" start-of-body= Jul 07 16:13:31.248913 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:31.248465 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" podUID="9f22bc65-3cff-4b9a-8ba9-52899e65a55a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": read tcp 10.132.0.2:33626->10.132.0.42:15021: read: connection reset by peer" Jul 07 16:13:32.232410 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:32.232370 2575 generic.go:356] "Generic (PLEG): container finished" podID="9f22bc65-3cff-4b9a-8ba9-52899e65a55a" containerID="04d666326b8113764f5413ccf1c10b73dd72a64d179bcec5e9650c81ea468368" exitCode=0 Jul 07 16:13:32.232586 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:32.232425 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" event={"ID":"9f22bc65-3cff-4b9a-8ba9-52899e65a55a","Type":"ContainerDied","Data":"04d666326b8113764f5413ccf1c10b73dd72a64d179bcec5e9650c81ea468368"} Jul 07 16:13:32.232830 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:32.232816 2575 scope.go:117] "RemoveContainer" containerID="04d666326b8113764f5413ccf1c10b73dd72a64d179bcec5e9650c81ea468368" Jul 07 16:13:32.233919 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:32.233891 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:13:32.233989 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:32.233963 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:13:32.234026 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:32.233997 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:13:33.238856 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:33.238820 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" event={"ID":"9f22bc65-3cff-4b9a-8ba9-52899e65a55a","Type":"ContainerStarted","Data":"33b9084eae9ddf30e0c5bd284a7a57d8f9337673711dae1ebe228167f33832a6"} Jul 07 16:13:33.594254 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:33.594228 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:33.594462 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:33.594438 2575 patch_prober.go:28] interesting pod/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 07 16:13:33.594543 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:33.594512 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" podUID="9f22bc65-3cff-4b9a-8ba9-52899e65a55a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 07 16:13:34.593962 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:34.593923 2575 patch_prober.go:28] interesting pod/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 07 16:13:34.594357 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:34.593985 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" podUID="9f22bc65-3cff-4b9a-8ba9-52899e65a55a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 07 16:13:35.594233 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:35.594195 2575 patch_prober.go:28] interesting pod/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 07 16:13:35.594663 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:35.594248 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" podUID="9f22bc65-3cff-4b9a-8ba9-52899e65a55a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 07 16:13:36.593919 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:36.593854 2575 patch_prober.go:28] interesting pod/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 07 16:13:36.594121 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:36.593939 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" podUID="9f22bc65-3cff-4b9a-8ba9-52899e65a55a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 07 16:13:37.597599 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:37.597569 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:37.598038 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:37.597822 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:13:37.598421 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:13:37.598406 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:14:06.775981 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:06.775946 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf"] Jul 07 16:14:06.776496 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:06.776151 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf" podUID="3b742bd3-0e73-4d07-b92a-85b6a33f5443" containerName="maas-api" containerID="cri-o://b4dc3f5cfdfcf5103a8fc35cb7de5e63090438c17d614397edd1d6d7d4bacd6f" gracePeriod=30 Jul 07 16:14:07.020265 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:07.020239 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf" Jul 07 16:14:07.076443 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:07.076368 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bwbh\" (UniqueName: \"kubernetes.io/projected/3b742bd3-0e73-4d07-b92a-85b6a33f5443-kube-api-access-4bwbh\") pod \"3b742bd3-0e73-4d07-b92a-85b6a33f5443\" (UID: \"3b742bd3-0e73-4d07-b92a-85b6a33f5443\") " Jul 07 16:14:07.076443 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:07.076402 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b742bd3-0e73-4d07-b92a-85b6a33f5443-maas-api-tls\") pod \"3b742bd3-0e73-4d07-b92a-85b6a33f5443\" (UID: \"3b742bd3-0e73-4d07-b92a-85b6a33f5443\") " Jul 07 16:14:07.076669 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:07.076480 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b742bd3-0e73-4d07-b92a-85b6a33f5443-openshift-service-ca\") pod \"3b742bd3-0e73-4d07-b92a-85b6a33f5443\" (UID: \"3b742bd3-0e73-4d07-b92a-85b6a33f5443\") " Jul 07 16:14:07.076825 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:07.076800 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b742bd3-0e73-4d07-b92a-85b6a33f5443-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3b742bd3-0e73-4d07-b92a-85b6a33f5443" (UID: "3b742bd3-0e73-4d07-b92a-85b6a33f5443"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:14:07.078563 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:07.078537 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b742bd3-0e73-4d07-b92a-85b6a33f5443-kube-api-access-4bwbh" (OuterVolumeSpecName: "kube-api-access-4bwbh") pod "3b742bd3-0e73-4d07-b92a-85b6a33f5443" (UID: "3b742bd3-0e73-4d07-b92a-85b6a33f5443"). InnerVolumeSpecName "kube-api-access-4bwbh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:14:07.078634 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:07.078544 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b742bd3-0e73-4d07-b92a-85b6a33f5443-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3b742bd3-0e73-4d07-b92a-85b6a33f5443" (UID: "3b742bd3-0e73-4d07-b92a-85b6a33f5443"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:14:07.177419 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:07.177380 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bwbh\" (UniqueName: \"kubernetes.io/projected/3b742bd3-0e73-4d07-b92a-85b6a33f5443-kube-api-access-4bwbh\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:14:07.177419 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:07.177414 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b742bd3-0e73-4d07-b92a-85b6a33f5443-maas-api-tls\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:14:07.177419 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:07.177423 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b742bd3-0e73-4d07-b92a-85b6a33f5443-openshift-service-ca\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:14:07.411797 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:07.411759 2575 generic.go:356] "Generic (PLEG): container finished" podID="3b742bd3-0e73-4d07-b92a-85b6a33f5443" containerID="b4dc3f5cfdfcf5103a8fc35cb7de5e63090438c17d614397edd1d6d7d4bacd6f" exitCode=0 Jul 07 16:14:07.412022 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:07.411806 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf" Jul 07 16:14:07.412022 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:07.411827 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf" event={"ID":"3b742bd3-0e73-4d07-b92a-85b6a33f5443","Type":"ContainerDied","Data":"b4dc3f5cfdfcf5103a8fc35cb7de5e63090438c17d614397edd1d6d7d4bacd6f"} Jul 07 16:14:07.412022 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:07.411906 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf" event={"ID":"3b742bd3-0e73-4d07-b92a-85b6a33f5443","Type":"ContainerDied","Data":"4a5499c06f43452353cf6675f703ec9ec63c545f8b104287f144bc972fa59e48"} Jul 07 16:14:07.412022 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:07.411937 2575 scope.go:117] "RemoveContainer" containerID="b4dc3f5cfdfcf5103a8fc35cb7de5e63090438c17d614397edd1d6d7d4bacd6f" Jul 07 16:14:07.422651 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:07.422634 2575 scope.go:117] "RemoveContainer" containerID="b4dc3f5cfdfcf5103a8fc35cb7de5e63090438c17d614397edd1d6d7d4bacd6f" Jul 07 16:14:07.423037 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:14:07.422985 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4dc3f5cfdfcf5103a8fc35cb7de5e63090438c17d614397edd1d6d7d4bacd6f\": container with ID starting with b4dc3f5cfdfcf5103a8fc35cb7de5e63090438c17d614397edd1d6d7d4bacd6f not found: ID does not exist" containerID="b4dc3f5cfdfcf5103a8fc35cb7de5e63090438c17d614397edd1d6d7d4bacd6f" Jul 07 16:14:07.423105 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:07.423051 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4dc3f5cfdfcf5103a8fc35cb7de5e63090438c17d614397edd1d6d7d4bacd6f"} err="failed to get container status \"b4dc3f5cfdfcf5103a8fc35cb7de5e63090438c17d614397edd1d6d7d4bacd6f\": rpc error: code = NotFound desc = could not find container \"b4dc3f5cfdfcf5103a8fc35cb7de5e63090438c17d614397edd1d6d7d4bacd6f\": container with ID starting with b4dc3f5cfdfcf5103a8fc35cb7de5e63090438c17d614397edd1d6d7d4bacd6f not found: ID does not exist" Jul 07 16:14:07.439829 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:07.439806 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf"] Jul 07 16:14:07.447574 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:07.447555 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-d4320a-57bd6d549d-tgfqf"] Jul 07 16:14:09.436079 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:09.436043 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b742bd3-0e73-4d07-b92a-85b6a33f5443" path="/var/lib/kubelet/pods/3b742bd3-0e73-4d07-b92a-85b6a33f5443/volumes" Jul 07 16:14:49.151417 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:49.151330 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc"] Jul 07 16:14:49.152035 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:49.151584 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" podUID="d5708e9b-da63-4c0e-81ef-67b35bf784e1" containerName="istio-proxy" containerID="cri-o://5b18d801af0255295c6ab770900611a4ba57b146740152c887227496f3b25996" gracePeriod=30 Jul 07 16:14:54.402478 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.402448 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:14:54.514623 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.514532 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istiod-ca-cert\") pod \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " Jul 07 16:14:54.514623 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.514608 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istio-envoy\") pod \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " Jul 07 16:14:54.514824 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.514652 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w275v\" (UniqueName: \"kubernetes.io/projected/d5708e9b-da63-4c0e-81ef-67b35bf784e1-kube-api-access-w275v\") pod \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " Jul 07 16:14:54.514824 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.514688 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-credential-socket\") pod \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " Jul 07 16:14:54.514824 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.514748 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istio-data\") pod \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " Jul 07 16:14:54.514824 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.514774 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istio-podinfo\") pod \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " Jul 07 16:14:54.515063 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.514824 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-workload-socket\") pod \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " Jul 07 16:14:54.515063 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.514859 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istio-token\") pod \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " Jul 07 16:14:54.515172 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.514990 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d5708e9b-da63-4c0e-81ef-67b35bf784e1" (UID: "d5708e9b-da63-4c0e-81ef-67b35bf784e1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:14:54.515172 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.515099 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d5708e9b-da63-4c0e-81ef-67b35bf784e1" (UID: "d5708e9b-da63-4c0e-81ef-67b35bf784e1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:14:54.515172 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.515112 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d5708e9b-da63-4c0e-81ef-67b35bf784e1" (UID: "d5708e9b-da63-4c0e-81ef-67b35bf784e1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:14:54.515172 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.515138 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-workload-certs\") pod \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\" (UID: \"d5708e9b-da63-4c0e-81ef-67b35bf784e1\") " Jul 07 16:14:54.515419 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.515398 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d5708e9b-da63-4c0e-81ef-67b35bf784e1" (UID: "d5708e9b-da63-4c0e-81ef-67b35bf784e1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:14:54.515557 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.515525 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-workload-socket\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:14:54.515557 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.515549 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-workload-certs\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:14:54.515751 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.515564 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istiod-ca-cert\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:14:54.515751 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.515577 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-credential-socket\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:14:54.515895 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.515764 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istio-data" (OuterVolumeSpecName: "istio-data") pod "d5708e9b-da63-4c0e-81ef-67b35bf784e1" (UID: "d5708e9b-da63-4c0e-81ef-67b35bf784e1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:14:54.517010 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.516990 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d5708e9b-da63-4c0e-81ef-67b35bf784e1" (UID: "d5708e9b-da63-4c0e-81ef-67b35bf784e1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:14:54.517093 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.517015 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istio-token" (OuterVolumeSpecName: "istio-token") pod "d5708e9b-da63-4c0e-81ef-67b35bf784e1" (UID: "d5708e9b-da63-4c0e-81ef-67b35bf784e1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:14:54.517093 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.517070 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5708e9b-da63-4c0e-81ef-67b35bf784e1-kube-api-access-w275v" (OuterVolumeSpecName: "kube-api-access-w275v") pod "d5708e9b-da63-4c0e-81ef-67b35bf784e1" (UID: "d5708e9b-da63-4c0e-81ef-67b35bf784e1"). InnerVolumeSpecName "kube-api-access-w275v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:14:54.517181 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.517164 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d5708e9b-da63-4c0e-81ef-67b35bf784e1" (UID: "d5708e9b-da63-4c0e-81ef-67b35bf784e1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 16:14:54.616561 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.616520 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istio-envoy\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:14:54.616561 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.616556 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w275v\" (UniqueName: \"kubernetes.io/projected/d5708e9b-da63-4c0e-81ef-67b35bf784e1-kube-api-access-w275v\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:14:54.616561 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.616566 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istio-data\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:14:54.616561 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.616575 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istio-podinfo\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:14:54.616853 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.616582 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5708e9b-da63-4c0e-81ef-67b35bf784e1-istio-token\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:14:54.638567 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.638541 2575 generic.go:356] "Generic (PLEG): container finished" podID="d5708e9b-da63-4c0e-81ef-67b35bf784e1" containerID="5b18d801af0255295c6ab770900611a4ba57b146740152c887227496f3b25996" exitCode=0 Jul 07 16:14:54.638697 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.638589 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" Jul 07 16:14:54.638697 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.638634 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" event={"ID":"d5708e9b-da63-4c0e-81ef-67b35bf784e1","Type":"ContainerDied","Data":"5b18d801af0255295c6ab770900611a4ba57b146740152c887227496f3b25996"} Jul 07 16:14:54.638697 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.638687 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc" event={"ID":"d5708e9b-da63-4c0e-81ef-67b35bf784e1","Type":"ContainerDied","Data":"36acf58c4d93e0ed3c7bf931db89c5efd81613902b5f8f24f2e0d2cf94613ac0"} Jul 07 16:14:54.638809 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.638710 2575 scope.go:117] "RemoveContainer" containerID="5b18d801af0255295c6ab770900611a4ba57b146740152c887227496f3b25996" Jul 07 16:14:54.649174 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.649148 2575 scope.go:117] "RemoveContainer" containerID="be4bf1cd614755b3e89f4843d92af4e87f0301e005a8d8b3660407f06d4c31af" Jul 07 16:14:54.658982 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.658962 2575 scope.go:117] "RemoveContainer" containerID="5b18d801af0255295c6ab770900611a4ba57b146740152c887227496f3b25996" Jul 07 16:14:54.659219 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:14:54.659202 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b18d801af0255295c6ab770900611a4ba57b146740152c887227496f3b25996\": container with ID starting with 5b18d801af0255295c6ab770900611a4ba57b146740152c887227496f3b25996 not found: ID does not exist" containerID="5b18d801af0255295c6ab770900611a4ba57b146740152c887227496f3b25996" Jul 07 16:14:54.659268 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.659225 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b18d801af0255295c6ab770900611a4ba57b146740152c887227496f3b25996"} err="failed to get container status \"5b18d801af0255295c6ab770900611a4ba57b146740152c887227496f3b25996\": rpc error: code = NotFound desc = could not find container \"5b18d801af0255295c6ab770900611a4ba57b146740152c887227496f3b25996\": container with ID starting with 5b18d801af0255295c6ab770900611a4ba57b146740152c887227496f3b25996 not found: ID does not exist" Jul 07 16:14:54.659268 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.659240 2575 scope.go:117] "RemoveContainer" containerID="be4bf1cd614755b3e89f4843d92af4e87f0301e005a8d8b3660407f06d4c31af" Jul 07 16:14:54.659464 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:14:54.659446 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be4bf1cd614755b3e89f4843d92af4e87f0301e005a8d8b3660407f06d4c31af\": container with ID starting with be4bf1cd614755b3e89f4843d92af4e87f0301e005a8d8b3660407f06d4c31af not found: ID does not exist" containerID="be4bf1cd614755b3e89f4843d92af4e87f0301e005a8d8b3660407f06d4c31af" Jul 07 16:14:54.659509 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.659471 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be4bf1cd614755b3e89f4843d92af4e87f0301e005a8d8b3660407f06d4c31af"} err="failed to get container status \"be4bf1cd614755b3e89f4843d92af4e87f0301e005a8d8b3660407f06d4c31af\": rpc error: code = NotFound desc = could not find container \"be4bf1cd614755b3e89f4843d92af4e87f0301e005a8d8b3660407f06d4c31af\": container with ID starting with be4bf1cd614755b3e89f4843d92af4e87f0301e005a8d8b3660407f06d4c31af not found: ID does not exist" Jul 07 16:14:54.671558 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.671535 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc"] Jul 07 16:14:54.676901 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:54.676881 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-d4320a-openshift-default-5589c89fb5-wphdc"] Jul 07 16:14:55.436359 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:55.436322 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5708e9b-da63-4c0e-81ef-67b35bf784e1" path="/var/lib/kubelet/pods/d5708e9b-da63-4c0e-81ef-67b35bf784e1/volumes" Jul 07 16:14:57.012104 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:57.012068 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6"] Jul 07 16:14:57.012511 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:57.012318 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" podUID="9f22bc65-3cff-4b9a-8ba9-52899e65a55a" containerName="istio-proxy" containerID="cri-o://33b9084eae9ddf30e0c5bd284a7a57d8f9337673711dae1ebe228167f33832a6" gracePeriod=30 Jul 07 16:14:57.947206 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:57.947176 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb"] Jul 07 16:14:57.947585 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:57.947568 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b742bd3-0e73-4d07-b92a-85b6a33f5443" containerName="maas-api" Jul 07 16:14:57.947641 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:57.947589 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b742bd3-0e73-4d07-b92a-85b6a33f5443" containerName="maas-api" Jul 07 16:14:57.947641 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:57.947627 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5708e9b-da63-4c0e-81ef-67b35bf784e1" containerName="istio-proxy" Jul 07 16:14:57.947641 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:57.947638 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5708e9b-da63-4c0e-81ef-67b35bf784e1" containerName="istio-proxy" Jul 07 16:14:57.947786 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:57.947660 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5708e9b-da63-4c0e-81ef-67b35bf784e1" containerName="istio-proxy" Jul 07 16:14:57.947786 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:57.947667 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5708e9b-da63-4c0e-81ef-67b35bf784e1" containerName="istio-proxy" Jul 07 16:14:57.947786 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:57.947756 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b742bd3-0e73-4d07-b92a-85b6a33f5443" containerName="maas-api" Jul 07 16:14:57.947786 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:57.947770 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5708e9b-da63-4c0e-81ef-67b35bf784e1" containerName="istio-proxy" Jul 07 16:14:57.947786 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:57.947779 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5708e9b-da63-4c0e-81ef-67b35bf784e1" containerName="istio-proxy" Jul 07 16:14:57.949764 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:57.949749 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:57.953962 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:57.953943 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-4c489a-openshift-default-dockercfg-wwthm\"" Jul 07 16:14:57.972652 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:57.972629 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb"] Jul 07 16:14:58.046553 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.046516 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx65g\" (UniqueName: \"kubernetes.io/projected/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-kube-api-access-bx65g\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.047039 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.046564 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istio-token\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.047039 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.046690 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istio-podinfo\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.047039 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.046774 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-workload-socket\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.047039 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.046812 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istiod-ca-cert\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.047039 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.046945 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-workload-certs\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.047309 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.047050 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istio-data\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.047309 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.047085 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-credential-socket\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.047309 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.047228 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istio-envoy\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.148123 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.148089 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-workload-socket\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.148298 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.148135 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istiod-ca-cert\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.148298 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.148173 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-workload-certs\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.148298 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.148205 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istio-data\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.148298 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.148234 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-credential-socket\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.148298 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.148282 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istio-envoy\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.148580 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.148345 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx65g\" (UniqueName: \"kubernetes.io/projected/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-kube-api-access-bx65g\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.148580 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.148380 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istio-token\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.148580 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.148535 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-workload-socket\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.148745 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.148591 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istio-podinfo\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.148745 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.148612 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-workload-certs\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.148745 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.148669 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istio-data\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.148929 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.148883 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istiod-ca-cert\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.148981 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.148923 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-credential-socket\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.150631 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.150609 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istio-envoy\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.150826 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.150809 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istio-podinfo\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.172498 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.172466 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istio-token\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.173451 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.173433 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx65g\" (UniqueName: \"kubernetes.io/projected/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-kube-api-access-bx65g\") pod \"e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.260157 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.260085 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:58.395854 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.395829 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb"] Jul 07 16:14:58.398432 ip-10-0-131-221 kubenswrapper[2575]: W0707 16:14:58.398405 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37abdaf0_40d4_48e8_9fe8_eeb9faf73cbc.slice/crio-644625f89e8ec77cce88c292b5d8448bd8b3fb28721d2f79985fffaf2bb32f61 WatchSource:0}: Error finding container 644625f89e8ec77cce88c292b5d8448bd8b3fb28721d2f79985fffaf2bb32f61: Status 404 returned error can't find the container with id 644625f89e8ec77cce88c292b5d8448bd8b3fb28721d2f79985fffaf2bb32f61 Jul 07 16:14:58.400448 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.400416 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:14:58.400532 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.400480 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:14:58.400532 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.400507 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:14:58.662584 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.662545 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" event={"ID":"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc","Type":"ContainerStarted","Data":"3c4398a7cd7024c180226fa4962ec83ea7e73cc0f579dfecab1f657810887ba2"} Jul 07 16:14:58.662753 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.662616 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" event={"ID":"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc","Type":"ContainerStarted","Data":"644625f89e8ec77cce88c292b5d8448bd8b3fb28721d2f79985fffaf2bb32f61"} Jul 07 16:14:58.691287 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:58.691227 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" podStartSLOduration=1.691205406 podStartE2EDuration="1.691205406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:14:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:14:58.686768074 +0000 UTC m=+1985.782049156" watchObservedRunningTime="2026-07-07 16:14:58.691205406 +0000 UTC m=+1985.786486488" Jul 07 16:14:59.260911 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:59.260876 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:59.265825 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:59.265801 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:59.667931 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:59.667901 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:14:59.668901 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:14:59.668881 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:15:00.135613 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:00.135578 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29724015-wlp5t"] Jul 07 16:15:00.138166 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:00.138148 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724015-wlp5t" Jul 07 16:15:00.140998 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:00.140983 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-5vqnq\"" Jul 07 16:15:00.150280 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:00.150258 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29724015-wlp5t"] Jul 07 16:15:00.270046 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:00.270000 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-768dh\" (UniqueName: \"kubernetes.io/projected/ab2222c4-c95c-418a-a6a0-29900026a514-kube-api-access-768dh\") pod \"maas-api-key-cleanup-29724015-wlp5t\" (UID: \"ab2222c4-c95c-418a-a6a0-29900026a514\") " pod="opendatahub/maas-api-key-cleanup-29724015-wlp5t" Jul 07 16:15:00.371502 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:00.371462 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-768dh\" (UniqueName: \"kubernetes.io/projected/ab2222c4-c95c-418a-a6a0-29900026a514-kube-api-access-768dh\") pod \"maas-api-key-cleanup-29724015-wlp5t\" (UID: \"ab2222c4-c95c-418a-a6a0-29900026a514\") " pod="opendatahub/maas-api-key-cleanup-29724015-wlp5t" Jul 07 16:15:00.384511 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:00.384485 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-768dh\" (UniqueName: \"kubernetes.io/projected/ab2222c4-c95c-418a-a6a0-29900026a514-kube-api-access-768dh\") pod \"maas-api-key-cleanup-29724015-wlp5t\" (UID: \"ab2222c4-c95c-418a-a6a0-29900026a514\") " pod="opendatahub/maas-api-key-cleanup-29724015-wlp5t" Jul 07 16:15:00.449669 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:00.449579 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724015-wlp5t" Jul 07 16:15:00.583438 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:00.583403 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29724015-wlp5t"] Jul 07 16:15:00.585307 ip-10-0-131-221 kubenswrapper[2575]: W0707 16:15:00.585281 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab2222c4_c95c_418a_a6a0_29900026a514.slice/crio-fc666f2102b21bdd1f6f71b86f540d51067d02833d8005b6ab20287b3cfb4f88 WatchSource:0}: Error finding container fc666f2102b21bdd1f6f71b86f540d51067d02833d8005b6ab20287b3cfb4f88: Status 404 returned error can't find the container with id fc666f2102b21bdd1f6f71b86f540d51067d02833d8005b6ab20287b3cfb4f88 Jul 07 16:15:00.673924 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:00.673886 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724015-wlp5t" event={"ID":"ab2222c4-c95c-418a-a6a0-29900026a514","Type":"ContainerStarted","Data":"fc666f2102b21bdd1f6f71b86f540d51067d02833d8005b6ab20287b3cfb4f88"} Jul 07 16:15:01.680134 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:01.680097 2575 generic.go:356] "Generic (PLEG): container finished" podID="ab2222c4-c95c-418a-a6a0-29900026a514" containerID="f9d8ada1ae2f58513380d950c5eb4080e7850a749be389e1302acad88f9e8620" exitCode=0 Jul 07 16:15:01.680548 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:01.680189 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724015-wlp5t" event={"ID":"ab2222c4-c95c-418a-a6a0-29900026a514","Type":"ContainerDied","Data":"f9d8ada1ae2f58513380d950c5eb4080e7850a749be389e1302acad88f9e8620"} Jul 07 16:15:02.267910 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.267887 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:15:02.391622 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.391589 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istio-data\") pod \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " Jul 07 16:15:02.391787 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.391645 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfspj\" (UniqueName: \"kubernetes.io/projected/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-kube-api-access-xfspj\") pod \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " Jul 07 16:15:02.391787 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.391669 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-workload-socket\") pod \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " Jul 07 16:15:02.391787 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.391699 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istiod-ca-cert\") pod \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " Jul 07 16:15:02.391787 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.391744 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istio-envoy\") pod \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " Jul 07 16:15:02.391787 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.391774 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istio-podinfo\") pod \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " Jul 07 16:15:02.392094 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.391796 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istio-token\") pod \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " Jul 07 16:15:02.392094 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.391830 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-credential-socket\") pod \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " Jul 07 16:15:02.392094 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.391924 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-workload-certs\") pod \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\" (UID: \"9f22bc65-3cff-4b9a-8ba9-52899e65a55a\") " Jul 07 16:15:02.392094 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.392029 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9f22bc65-3cff-4b9a-8ba9-52899e65a55a" (UID: "9f22bc65-3cff-4b9a-8ba9-52899e65a55a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:15:02.392398 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.392253 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9f22bc65-3cff-4b9a-8ba9-52899e65a55a" (UID: "9f22bc65-3cff-4b9a-8ba9-52899e65a55a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:15:02.392398 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.392340 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-workload-socket\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:02.392550 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.392399 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istio-data" (OuterVolumeSpecName: "istio-data") pod "9f22bc65-3cff-4b9a-8ba9-52899e65a55a" (UID: "9f22bc65-3cff-4b9a-8ba9-52899e65a55a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:15:02.392550 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.392422 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9f22bc65-3cff-4b9a-8ba9-52899e65a55a" (UID: "9f22bc65-3cff-4b9a-8ba9-52899e65a55a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:15:02.392550 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.392521 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9f22bc65-3cff-4b9a-8ba9-52899e65a55a" (UID: "9f22bc65-3cff-4b9a-8ba9-52899e65a55a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:15:02.394159 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.394140 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istio-token" (OuterVolumeSpecName: "istio-token") pod "9f22bc65-3cff-4b9a-8ba9-52899e65a55a" (UID: "9f22bc65-3cff-4b9a-8ba9-52899e65a55a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:15:02.394159 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.394146 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9f22bc65-3cff-4b9a-8ba9-52899e65a55a" (UID: "9f22bc65-3cff-4b9a-8ba9-52899e65a55a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:15:02.394441 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.394419 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9f22bc65-3cff-4b9a-8ba9-52899e65a55a" (UID: "9f22bc65-3cff-4b9a-8ba9-52899e65a55a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 16:15:02.394441 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.394429 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-kube-api-access-xfspj" (OuterVolumeSpecName: "kube-api-access-xfspj") pod "9f22bc65-3cff-4b9a-8ba9-52899e65a55a" (UID: "9f22bc65-3cff-4b9a-8ba9-52899e65a55a"). InnerVolumeSpecName "kube-api-access-xfspj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:15:02.493010 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.492977 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfspj\" (UniqueName: \"kubernetes.io/projected/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-kube-api-access-xfspj\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:02.493010 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.493010 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istiod-ca-cert\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:02.493222 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.493022 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istio-envoy\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:02.493222 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.493034 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istio-podinfo\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:02.493222 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.493044 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istio-token\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:02.493222 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.493055 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-credential-socket\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:02.493222 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.493065 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-workload-certs\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:02.493222 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.493077 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f22bc65-3cff-4b9a-8ba9-52899e65a55a-istio-data\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:02.686427 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.686329 2575 generic.go:356] "Generic (PLEG): container finished" podID="9f22bc65-3cff-4b9a-8ba9-52899e65a55a" containerID="33b9084eae9ddf30e0c5bd284a7a57d8f9337673711dae1ebe228167f33832a6" exitCode=0 Jul 07 16:15:02.686427 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.686380 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" Jul 07 16:15:02.686427 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.686421 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" event={"ID":"9f22bc65-3cff-4b9a-8ba9-52899e65a55a","Type":"ContainerDied","Data":"33b9084eae9ddf30e0c5bd284a7a57d8f9337673711dae1ebe228167f33832a6"} Jul 07 16:15:02.686995 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.686474 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6" event={"ID":"9f22bc65-3cff-4b9a-8ba9-52899e65a55a","Type":"ContainerDied","Data":"d26b1238b3a3b17a90105a5784003741c69e1bb7a2c6d5115dc8c2e247e20183"} Jul 07 16:15:02.686995 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.686503 2575 scope.go:117] "RemoveContainer" containerID="33b9084eae9ddf30e0c5bd284a7a57d8f9337673711dae1ebe228167f33832a6" Jul 07 16:15:02.695851 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.695831 2575 scope.go:117] "RemoveContainer" containerID="04d666326b8113764f5413ccf1c10b73dd72a64d179bcec5e9650c81ea468368" Jul 07 16:15:02.706064 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.706044 2575 scope.go:117] "RemoveContainer" containerID="33b9084eae9ddf30e0c5bd284a7a57d8f9337673711dae1ebe228167f33832a6" Jul 07 16:15:02.706346 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:15:02.706318 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33b9084eae9ddf30e0c5bd284a7a57d8f9337673711dae1ebe228167f33832a6\": container with ID starting with 33b9084eae9ddf30e0c5bd284a7a57d8f9337673711dae1ebe228167f33832a6 not found: ID does not exist" containerID="33b9084eae9ddf30e0c5bd284a7a57d8f9337673711dae1ebe228167f33832a6" Jul 07 16:15:02.706451 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.706352 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33b9084eae9ddf30e0c5bd284a7a57d8f9337673711dae1ebe228167f33832a6"} err="failed to get container status \"33b9084eae9ddf30e0c5bd284a7a57d8f9337673711dae1ebe228167f33832a6\": rpc error: code = NotFound desc = could not find container \"33b9084eae9ddf30e0c5bd284a7a57d8f9337673711dae1ebe228167f33832a6\": container with ID starting with 33b9084eae9ddf30e0c5bd284a7a57d8f9337673711dae1ebe228167f33832a6 not found: ID does not exist" Jul 07 16:15:02.706451 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.706378 2575 scope.go:117] "RemoveContainer" containerID="04d666326b8113764f5413ccf1c10b73dd72a64d179bcec5e9650c81ea468368" Jul 07 16:15:02.706637 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:15:02.706617 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04d666326b8113764f5413ccf1c10b73dd72a64d179bcec5e9650c81ea468368\": container with ID starting with 04d666326b8113764f5413ccf1c10b73dd72a64d179bcec5e9650c81ea468368 not found: ID does not exist" containerID="04d666326b8113764f5413ccf1c10b73dd72a64d179bcec5e9650c81ea468368" Jul 07 16:15:02.706683 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.706640 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04d666326b8113764f5413ccf1c10b73dd72a64d179bcec5e9650c81ea468368"} err="failed to get container status \"04d666326b8113764f5413ccf1c10b73dd72a64d179bcec5e9650c81ea468368\": rpc error: code = NotFound desc = could not find container \"04d666326b8113764f5413ccf1c10b73dd72a64d179bcec5e9650c81ea468368\": container with ID starting with 04d666326b8113764f5413ccf1c10b73dd72a64d179bcec5e9650c81ea468368 not found: ID does not exist" Jul 07 16:15:02.760385 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.760358 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6"] Jul 07 16:15:02.774093 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.774064 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-eb74cf-openshift-default-b966865d5-kpdp6"] Jul 07 16:15:02.822984 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.822956 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724015-wlp5t" Jul 07 16:15:02.897627 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.897592 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-768dh\" (UniqueName: \"kubernetes.io/projected/ab2222c4-c95c-418a-a6a0-29900026a514-kube-api-access-768dh\") pod \"ab2222c4-c95c-418a-a6a0-29900026a514\" (UID: \"ab2222c4-c95c-418a-a6a0-29900026a514\") " Jul 07 16:15:02.899670 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.899644 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab2222c4-c95c-418a-a6a0-29900026a514-kube-api-access-768dh" (OuterVolumeSpecName: "kube-api-access-768dh") pod "ab2222c4-c95c-418a-a6a0-29900026a514" (UID: "ab2222c4-c95c-418a-a6a0-29900026a514"). InnerVolumeSpecName "kube-api-access-768dh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:15:02.999104 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:02.999015 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-768dh\" (UniqueName: \"kubernetes.io/projected/ab2222c4-c95c-418a-a6a0-29900026a514-kube-api-access-768dh\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:03.436324 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:03.436283 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f22bc65-3cff-4b9a-8ba9-52899e65a55a" path="/var/lib/kubelet/pods/9f22bc65-3cff-4b9a-8ba9-52899e65a55a/volumes" Jul 07 16:15:03.694205 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:03.694127 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724015-wlp5t" Jul 07 16:15:03.694205 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:03.694140 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724015-wlp5t" event={"ID":"ab2222c4-c95c-418a-a6a0-29900026a514","Type":"ContainerDied","Data":"fc666f2102b21bdd1f6f71b86f540d51067d02833d8005b6ab20287b3cfb4f88"} Jul 07 16:15:03.694205 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:03.694183 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc666f2102b21bdd1f6f71b86f540d51067d02833d8005b6ab20287b3cfb4f88" Jul 07 16:15:05.938919 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:05.938886 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246"] Jul 07 16:15:05.939324 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:05.939290 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f22bc65-3cff-4b9a-8ba9-52899e65a55a" containerName="istio-proxy" Jul 07 16:15:05.939324 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:05.939302 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f22bc65-3cff-4b9a-8ba9-52899e65a55a" containerName="istio-proxy" Jul 07 16:15:05.939324 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:05.939309 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab2222c4-c95c-418a-a6a0-29900026a514" containerName="cleanup" Jul 07 16:15:05.939324 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:05.939314 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab2222c4-c95c-418a-a6a0-29900026a514" containerName="cleanup" Jul 07 16:15:05.939458 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:05.939332 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f22bc65-3cff-4b9a-8ba9-52899e65a55a" containerName="istio-proxy" Jul 07 16:15:05.939458 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:05.939339 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f22bc65-3cff-4b9a-8ba9-52899e65a55a" containerName="istio-proxy" Jul 07 16:15:05.939458 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:05.939402 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f22bc65-3cff-4b9a-8ba9-52899e65a55a" containerName="istio-proxy" Jul 07 16:15:05.939458 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:05.939412 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f22bc65-3cff-4b9a-8ba9-52899e65a55a" containerName="istio-proxy" Jul 07 16:15:05.939458 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:05.939418 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab2222c4-c95c-418a-a6a0-29900026a514" containerName="cleanup" Jul 07 16:15:05.941972 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:05.941954 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:05.945258 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:05.945240 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-b155c0-openshift-default-dockercfg-zvwfw\"" Jul 07 16:15:05.957815 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:05.957790 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246"] Jul 07 16:15:06.027874 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.027838 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/769bd66f-e164-483e-88b5-d2a136af3f11-istio-podinfo\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.028085 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.027896 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-workload-socket\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.028085 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.027952 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg59p\" (UniqueName: \"kubernetes.io/projected/769bd66f-e164-483e-88b5-d2a136af3f11-kube-api-access-mg59p\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.028085 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.028013 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-credential-socket\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.028085 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.028046 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-istio-envoy\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.028085 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.028066 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/769bd66f-e164-483e-88b5-d2a136af3f11-istio-token\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.028357 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.028115 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/769bd66f-e164-483e-88b5-d2a136af3f11-istiod-ca-cert\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.028357 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.028167 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-workload-certs\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.028357 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.028192 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-istio-data\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.129283 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.129243 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-istio-data\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.129469 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.129305 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/769bd66f-e164-483e-88b5-d2a136af3f11-istio-podinfo\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.129469 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.129324 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-workload-socket\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.129469 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.129355 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg59p\" (UniqueName: \"kubernetes.io/projected/769bd66f-e164-483e-88b5-d2a136af3f11-kube-api-access-mg59p\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.129469 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.129381 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-credential-socket\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.129469 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.129399 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-istio-envoy\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.129469 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.129413 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/769bd66f-e164-483e-88b5-d2a136af3f11-istio-token\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.129469 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.129446 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/769bd66f-e164-483e-88b5-d2a136af3f11-istiod-ca-cert\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.129813 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.129506 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-workload-certs\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.130164 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.130135 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-workload-socket\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.130396 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.130330 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-credential-socket\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.130396 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.130353 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-workload-certs\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.130558 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.129701 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-istio-data\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.130776 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.130750 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/769bd66f-e164-483e-88b5-d2a136af3f11-istiod-ca-cert\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.137405 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.132731 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-istio-envoy\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.137405 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.132943 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/769bd66f-e164-483e-88b5-d2a136af3f11-istio-podinfo\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.143281 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.143259 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg59p\" (UniqueName: \"kubernetes.io/projected/769bd66f-e164-483e-88b5-d2a136af3f11-kube-api-access-mg59p\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.143846 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.143820 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/769bd66f-e164-483e-88b5-d2a136af3f11-istio-token\") pod \"e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.251929 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.251825 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:06.395096 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.395063 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246"] Jul 07 16:15:06.396566 ip-10-0-131-221 kubenswrapper[2575]: W0707 16:15:06.396532 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod769bd66f_e164_483e_88b5_d2a136af3f11.slice/crio-8048be5497244cdb257cfa03b9cc4d7c0135e6a32ef32a70cedee4214a816c4f WatchSource:0}: Error finding container 8048be5497244cdb257cfa03b9cc4d7c0135e6a32ef32a70cedee4214a816c4f: Status 404 returned error can't find the container with id 8048be5497244cdb257cfa03b9cc4d7c0135e6a32ef32a70cedee4214a816c4f Jul 07 16:15:06.398649 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.398613 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:15:06.398721 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.398678 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:15:06.398721 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.398709 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:15:06.710283 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.710249 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" event={"ID":"769bd66f-e164-483e-88b5-d2a136af3f11","Type":"ContainerStarted","Data":"b7b7f0f3d39b8bf5a811158bf7dfc902c50fbd882b8e7c8e9ef739ce5292a882"} Jul 07 16:15:06.710466 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.710306 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" event={"ID":"769bd66f-e164-483e-88b5-d2a136af3f11","Type":"ContainerStarted","Data":"8048be5497244cdb257cfa03b9cc4d7c0135e6a32ef32a70cedee4214a816c4f"} Jul 07 16:15:06.746643 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:06.746586 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" podStartSLOduration=1.746570569 podStartE2EDuration="1.746570569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:15:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:15:06.744930342 +0000 UTC m=+1993.840211420" watchObservedRunningTime="2026-07-07 16:15:06.746570569 +0000 UTC m=+1993.841851648" Jul 07 16:15:07.252778 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:07.252741 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:07.257525 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:07.257502 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:07.717227 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:07.717191 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:07.718150 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:07.718127 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:08.763429 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:08.763382 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf"] Jul 07 16:15:08.768827 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:08.768800 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf" Jul 07 16:15:08.772481 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:08.772451 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-b155c0-serving-cert\"" Jul 07 16:15:08.772602 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:08.772488 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-5vqnq\"" Jul 07 16:15:08.781841 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:08.781817 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf"] Jul 07 16:15:08.864660 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:08.864616 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4e0ac2f-3992-4903-934a-616f68d11d99-openshift-service-ca\") pod \"maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf\" (UID: \"a4e0ac2f-3992-4903-934a-616f68d11d99\") " pod="opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf" Jul 07 16:15:08.864842 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:08.864687 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4e0ac2f-3992-4903-934a-616f68d11d99-maas-api-tls\") pod \"maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf\" (UID: \"a4e0ac2f-3992-4903-934a-616f68d11d99\") " pod="opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf" Jul 07 16:15:08.864842 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:08.864756 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm68b\" (UniqueName: \"kubernetes.io/projected/a4e0ac2f-3992-4903-934a-616f68d11d99-kube-api-access-xm68b\") pod \"maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf\" (UID: \"a4e0ac2f-3992-4903-934a-616f68d11d99\") " pod="opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf" Jul 07 16:15:08.965828 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:08.965775 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4e0ac2f-3992-4903-934a-616f68d11d99-openshift-service-ca\") pod \"maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf\" (UID: \"a4e0ac2f-3992-4903-934a-616f68d11d99\") " pod="opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf" Jul 07 16:15:08.965828 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:08.965835 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4e0ac2f-3992-4903-934a-616f68d11d99-maas-api-tls\") pod \"maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf\" (UID: \"a4e0ac2f-3992-4903-934a-616f68d11d99\") " pod="opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf" Jul 07 16:15:08.966129 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:08.965888 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm68b\" (UniqueName: \"kubernetes.io/projected/a4e0ac2f-3992-4903-934a-616f68d11d99-kube-api-access-xm68b\") pod \"maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf\" (UID: \"a4e0ac2f-3992-4903-934a-616f68d11d99\") " pod="opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf" Jul 07 16:15:08.966129 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:15:08.966015 2575 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-b155c0-serving-cert: secret "maas-api-e2e-api-b-b155c0-serving-cert" not found Jul 07 16:15:08.966129 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:15:08.966119 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4e0ac2f-3992-4903-934a-616f68d11d99-maas-api-tls podName:a4e0ac2f-3992-4903-934a-616f68d11d99 nodeName:}" failed. No retries permitted until 2026-07-07 16:15:09.466095159 +0000 UTC m=+1996.561376236 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a4e0ac2f-3992-4903-934a-616f68d11d99-maas-api-tls") pod "maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf" (UID: "a4e0ac2f-3992-4903-934a-616f68d11d99") : secret "maas-api-e2e-api-b-b155c0-serving-cert" not found Jul 07 16:15:08.966493 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:08.966471 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4e0ac2f-3992-4903-934a-616f68d11d99-openshift-service-ca\") pod \"maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf\" (UID: \"a4e0ac2f-3992-4903-934a-616f68d11d99\") " pod="opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf" Jul 07 16:15:08.977499 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:08.977471 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm68b\" (UniqueName: \"kubernetes.io/projected/a4e0ac2f-3992-4903-934a-616f68d11d99-kube-api-access-xm68b\") pod \"maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf\" (UID: \"a4e0ac2f-3992-4903-934a-616f68d11d99\") " pod="opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf" Jul 07 16:15:09.471414 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:09.471371 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4e0ac2f-3992-4903-934a-616f68d11d99-maas-api-tls\") pod \"maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf\" (UID: \"a4e0ac2f-3992-4903-934a-616f68d11d99\") " pod="opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf" Jul 07 16:15:09.471577 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:15:09.471526 2575 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-b155c0-serving-cert: secret "maas-api-e2e-api-b-b155c0-serving-cert" not found Jul 07 16:15:09.471616 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:15:09.471588 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4e0ac2f-3992-4903-934a-616f68d11d99-maas-api-tls podName:a4e0ac2f-3992-4903-934a-616f68d11d99 nodeName:}" failed. No retries permitted until 2026-07-07 16:15:10.47157324 +0000 UTC m=+1997.566854298 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a4e0ac2f-3992-4903-934a-616f68d11d99-maas-api-tls") pod "maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf" (UID: "a4e0ac2f-3992-4903-934a-616f68d11d99") : secret "maas-api-e2e-api-b-b155c0-serving-cert" not found Jul 07 16:15:10.480114 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:10.480070 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4e0ac2f-3992-4903-934a-616f68d11d99-maas-api-tls\") pod \"maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf\" (UID: \"a4e0ac2f-3992-4903-934a-616f68d11d99\") " pod="opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf" Jul 07 16:15:10.482434 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:10.482406 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4e0ac2f-3992-4903-934a-616f68d11d99-maas-api-tls\") pod \"maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf\" (UID: \"a4e0ac2f-3992-4903-934a-616f68d11d99\") " pod="opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf" Jul 07 16:15:10.581322 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:10.581285 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf" Jul 07 16:15:10.734394 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:10.734365 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf"] Jul 07 16:15:10.735721 ip-10-0-131-221 kubenswrapper[2575]: W0707 16:15:10.735698 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4e0ac2f_3992_4903_934a_616f68d11d99.slice/crio-5bbde009131dca020d75fa036e00a3b6200eae8e935d2dcad73eeb4806c24b0e WatchSource:0}: Error finding container 5bbde009131dca020d75fa036e00a3b6200eae8e935d2dcad73eeb4806c24b0e: Status 404 returned error can't find the container with id 5bbde009131dca020d75fa036e00a3b6200eae8e935d2dcad73eeb4806c24b0e Jul 07 16:15:11.740391 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:11.740346 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf" event={"ID":"a4e0ac2f-3992-4903-934a-616f68d11d99","Type":"ContainerStarted","Data":"b368775f098b6fcee13a6074eddb4003257bbb5ed75455bff73448dba2252b52"} Jul 07 16:15:11.740858 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:11.740412 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf" event={"ID":"a4e0ac2f-3992-4903-934a-616f68d11d99","Type":"ContainerStarted","Data":"5bbde009131dca020d75fa036e00a3b6200eae8e935d2dcad73eeb4806c24b0e"} Jul 07 16:15:11.741051 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:11.741025 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf" Jul 07 16:15:11.769286 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:11.769242 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf" podStartSLOduration=3.351494689 podStartE2EDuration="3.769226111s" totalImagesPullingTime="417.731422ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:15:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:15:11.768243199 +0000 UTC m=+1998.863524279" watchObservedRunningTime="2026-07-07 16:15:11.769226111 +0000 UTC m=+1998.864507190" Jul 07 16:15:15.235104 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:15.234947 2575 patch_prober.go:28] interesting pod/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": read tcp 10.132.0.2:52356->10.132.0.43:15021: read: connection reset by peer" start-of-body= Jul 07 16:15:15.235104 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:15.235010 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" podUID="37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": read tcp 10.132.0.2:52356->10.132.0.43:15021: read: connection reset by peer" Jul 07 16:15:15.765532 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:15.765489 2575 generic.go:356] "Generic (PLEG): container finished" podID="37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc" containerID="3c4398a7cd7024c180226fa4962ec83ea7e73cc0f579dfecab1f657810887ba2" exitCode=0 Jul 07 16:15:15.765914 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:15.765783 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" event={"ID":"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc","Type":"ContainerDied","Data":"3c4398a7cd7024c180226fa4962ec83ea7e73cc0f579dfecab1f657810887ba2"} Jul 07 16:15:15.766463 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:15.766445 2575 scope.go:117] "RemoveContainer" containerID="3c4398a7cd7024c180226fa4962ec83ea7e73cc0f579dfecab1f657810887ba2" Jul 07 16:15:15.767567 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:15.767537 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:15:15.767656 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:15.767622 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:15:15.767729 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:15.767673 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:15:16.775288 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:16.775247 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" event={"ID":"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc","Type":"ContainerStarted","Data":"8192c9d6f6c8bf662136e444d5a3385044f95a22fb2f0751963a6f2941899590"} Jul 07 16:15:17.261213 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:17.260761 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:15:17.261213 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:17.260905 2575 patch_prober.go:28] interesting pod/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 07 16:15:17.261213 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:17.260949 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" podUID="37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 07 16:15:18.261344 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:18.261296 2575 patch_prober.go:28] interesting pod/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 07 16:15:18.261903 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:18.261367 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" podUID="37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 07 16:15:18.763782 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:18.763739 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf" Jul 07 16:15:19.260661 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:19.260616 2575 patch_prober.go:28] interesting pod/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 07 16:15:19.260922 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:19.260683 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" podUID="37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 07 16:15:20.261379 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:20.261286 2575 patch_prober.go:28] interesting pod/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 07 16:15:20.262355 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:20.261932 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" podUID="37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 07 16:15:21.265444 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:21.265404 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:15:21.265945 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:21.265712 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:15:21.266419 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:21.266401 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:15:22.879310 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:22.878977 2575 patch_prober.go:28] interesting pod/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": read tcp 10.132.0.2:48368->10.132.0.45:15021: read: connection reset by peer" start-of-body= Jul 07 16:15:22.879310 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:22.879056 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" podUID="769bd66f-e164-483e-88b5-d2a136af3f11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": read tcp 10.132.0.2:48368->10.132.0.45:15021: read: connection reset by peer" Jul 07 16:15:23.817528 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:23.817493 2575 generic.go:356] "Generic (PLEG): container finished" podID="769bd66f-e164-483e-88b5-d2a136af3f11" containerID="b7b7f0f3d39b8bf5a811158bf7dfc902c50fbd882b8e7c8e9ef739ce5292a882" exitCode=0 Jul 07 16:15:23.817707 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:23.817565 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" event={"ID":"769bd66f-e164-483e-88b5-d2a136af3f11","Type":"ContainerDied","Data":"b7b7f0f3d39b8bf5a811158bf7dfc902c50fbd882b8e7c8e9ef739ce5292a882"} Jul 07 16:15:23.818001 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:23.817987 2575 scope.go:117] "RemoveContainer" containerID="b7b7f0f3d39b8bf5a811158bf7dfc902c50fbd882b8e7c8e9ef739ce5292a882" Jul 07 16:15:23.819030 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:23.819002 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:15:23.819114 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:23.819073 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:15:23.819114 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:23.819109 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:15:24.824328 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:24.824298 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" event={"ID":"769bd66f-e164-483e-88b5-d2a136af3f11","Type":"ContainerStarted","Data":"275dfb595ccdb3d3043f3740c8e7d1231ea5b3a031aa3cea711b73dde2851d98"} Jul 07 16:15:25.253201 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:25.253163 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:25.253722 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:25.253625 2575 patch_prober.go:28] interesting pod/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 07 16:15:25.253939 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:25.253915 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" podUID="769bd66f-e164-483e-88b5-d2a136af3f11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 07 16:15:26.252581 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:26.252542 2575 patch_prober.go:28] interesting pod/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 07 16:15:26.253209 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:26.253175 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" podUID="769bd66f-e164-483e-88b5-d2a136af3f11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 07 16:15:27.252729 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:27.252689 2575 patch_prober.go:28] interesting pod/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 07 16:15:27.253338 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:27.253302 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" podUID="769bd66f-e164-483e-88b5-d2a136af3f11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 07 16:15:28.253568 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:28.253125 2575 patch_prober.go:28] interesting pod/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 07 16:15:28.253568 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:28.253191 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" podUID="769bd66f-e164-483e-88b5-d2a136af3f11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 07 16:15:29.256133 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:29.256106 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:29.256530 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:29.256361 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:29.256968 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:29.256950 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:29.393876 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:29.393831 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb"] Jul 07 16:15:29.394061 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:29.394039 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" podUID="37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc" containerName="istio-proxy" containerID="cri-o://8192c9d6f6c8bf662136e444d5a3385044f95a22fb2f0751963a6f2941899590" gracePeriod=30 Jul 07 16:15:30.544326 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:30.544291 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf"] Jul 07 16:15:30.544732 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:30.544515 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf" podUID="a4e0ac2f-3992-4903-934a-616f68d11d99" containerName="maas-api" containerID="cri-o://b368775f098b6fcee13a6074eddb4003257bbb5ed75455bff73448dba2252b52" gracePeriod=30 Jul 07 16:15:30.789900 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:30.789876 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf" Jul 07 16:15:30.855365 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:30.855335 2575 generic.go:356] "Generic (PLEG): container finished" podID="a4e0ac2f-3992-4903-934a-616f68d11d99" containerID="b368775f098b6fcee13a6074eddb4003257bbb5ed75455bff73448dba2252b52" exitCode=0 Jul 07 16:15:30.855525 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:30.855378 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf" Jul 07 16:15:30.855525 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:30.855417 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf" event={"ID":"a4e0ac2f-3992-4903-934a-616f68d11d99","Type":"ContainerDied","Data":"b368775f098b6fcee13a6074eddb4003257bbb5ed75455bff73448dba2252b52"} Jul 07 16:15:30.855525 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:30.855467 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf" event={"ID":"a4e0ac2f-3992-4903-934a-616f68d11d99","Type":"ContainerDied","Data":"5bbde009131dca020d75fa036e00a3b6200eae8e935d2dcad73eeb4806c24b0e"} Jul 07 16:15:30.855525 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:30.855488 2575 scope.go:117] "RemoveContainer" containerID="b368775f098b6fcee13a6074eddb4003257bbb5ed75455bff73448dba2252b52" Jul 07 16:15:30.864090 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:30.864074 2575 scope.go:117] "RemoveContainer" containerID="b368775f098b6fcee13a6074eddb4003257bbb5ed75455bff73448dba2252b52" Jul 07 16:15:30.864360 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:15:30.864341 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b368775f098b6fcee13a6074eddb4003257bbb5ed75455bff73448dba2252b52\": container with ID starting with b368775f098b6fcee13a6074eddb4003257bbb5ed75455bff73448dba2252b52 not found: ID does not exist" containerID="b368775f098b6fcee13a6074eddb4003257bbb5ed75455bff73448dba2252b52" Jul 07 16:15:30.864423 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:30.864373 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b368775f098b6fcee13a6074eddb4003257bbb5ed75455bff73448dba2252b52"} err="failed to get container status \"b368775f098b6fcee13a6074eddb4003257bbb5ed75455bff73448dba2252b52\": rpc error: code = NotFound desc = could not find container \"b368775f098b6fcee13a6074eddb4003257bbb5ed75455bff73448dba2252b52\": container with ID starting with b368775f098b6fcee13a6074eddb4003257bbb5ed75455bff73448dba2252b52 not found: ID does not exist" Jul 07 16:15:30.898705 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:30.898677 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4e0ac2f-3992-4903-934a-616f68d11d99-maas-api-tls\") pod \"a4e0ac2f-3992-4903-934a-616f68d11d99\" (UID: \"a4e0ac2f-3992-4903-934a-616f68d11d99\") " Jul 07 16:15:30.898806 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:30.898759 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xm68b\" (UniqueName: \"kubernetes.io/projected/a4e0ac2f-3992-4903-934a-616f68d11d99-kube-api-access-xm68b\") pod \"a4e0ac2f-3992-4903-934a-616f68d11d99\" (UID: \"a4e0ac2f-3992-4903-934a-616f68d11d99\") " Jul 07 16:15:30.898846 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:30.898806 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4e0ac2f-3992-4903-934a-616f68d11d99-openshift-service-ca\") pod \"a4e0ac2f-3992-4903-934a-616f68d11d99\" (UID: \"a4e0ac2f-3992-4903-934a-616f68d11d99\") " Jul 07 16:15:30.899244 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:30.899206 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4e0ac2f-3992-4903-934a-616f68d11d99-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a4e0ac2f-3992-4903-934a-616f68d11d99" (UID: "a4e0ac2f-3992-4903-934a-616f68d11d99"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:15:30.900840 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:30.900818 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4e0ac2f-3992-4903-934a-616f68d11d99-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a4e0ac2f-3992-4903-934a-616f68d11d99" (UID: "a4e0ac2f-3992-4903-934a-616f68d11d99"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:15:30.900936 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:30.900827 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4e0ac2f-3992-4903-934a-616f68d11d99-kube-api-access-xm68b" (OuterVolumeSpecName: "kube-api-access-xm68b") pod "a4e0ac2f-3992-4903-934a-616f68d11d99" (UID: "a4e0ac2f-3992-4903-934a-616f68d11d99"). InnerVolumeSpecName "kube-api-access-xm68b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:15:31.000281 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:31.000246 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4e0ac2f-3992-4903-934a-616f68d11d99-maas-api-tls\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:31.000281 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:31.000279 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xm68b\" (UniqueName: \"kubernetes.io/projected/a4e0ac2f-3992-4903-934a-616f68d11d99-kube-api-access-xm68b\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:31.000281 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:31.000289 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4e0ac2f-3992-4903-934a-616f68d11d99-openshift-service-ca\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:31.179696 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:31.179664 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf"] Jul 07 16:15:31.184582 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:31.184555 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-b155c0-6b478f4c74-86ccf"] Jul 07 16:15:31.435965 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:31.435888 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4e0ac2f-3992-4903-934a-616f68d11d99" path="/var/lib/kubelet/pods/a4e0ac2f-3992-4903-934a-616f68d11d99/volumes" Jul 07 16:15:34.651041 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.651017 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:15:34.735399 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.735332 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istio-podinfo\") pod \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " Jul 07 16:15:34.735399 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.735366 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istiod-ca-cert\") pod \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " Jul 07 16:15:34.735399 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.735395 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istio-data\") pod \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " Jul 07 16:15:34.735616 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.735441 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istio-envoy\") pod \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " Jul 07 16:15:34.735616 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.735455 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-workload-certs\") pod \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " Jul 07 16:15:34.735616 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.735475 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bx65g\" (UniqueName: \"kubernetes.io/projected/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-kube-api-access-bx65g\") pod \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " Jul 07 16:15:34.735616 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.735522 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-workload-socket\") pod \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " Jul 07 16:15:34.735616 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.735557 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-credential-socket\") pod \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " Jul 07 16:15:34.735616 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.735612 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istio-token\") pod \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\" (UID: \"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc\") " Jul 07 16:15:34.735957 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.735759 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc" (UID: "37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:15:34.736020 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.736004 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-workload-certs\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:34.736081 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.736018 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc" (UID: "37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:15:34.736259 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.736220 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc" (UID: "37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:15:34.736259 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.736238 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc" (UID: "37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:15:34.736683 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.736455 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istio-data" (OuterVolumeSpecName: "istio-data") pod "37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc" (UID: "37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:15:34.737632 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.737607 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc" (UID: "37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 16:15:34.737762 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.737735 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-kube-api-access-bx65g" (OuterVolumeSpecName: "kube-api-access-bx65g") pod "37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc" (UID: "37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc"). InnerVolumeSpecName "kube-api-access-bx65g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:15:34.738036 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.738014 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc" (UID: "37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:15:34.738128 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.738031 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istio-token" (OuterVolumeSpecName: "istio-token") pod "37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc" (UID: "37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:15:34.837233 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.837208 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istio-envoy\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:34.837233 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.837231 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bx65g\" (UniqueName: \"kubernetes.io/projected/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-kube-api-access-bx65g\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:34.837392 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.837243 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-workload-socket\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:34.837392 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.837253 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-credential-socket\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:34.837392 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.837262 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istio-token\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:34.837392 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.837270 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istio-podinfo\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:34.837392 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.837277 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istiod-ca-cert\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:34.837392 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.837285 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc-istio-data\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:34.880250 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.880219 2575 generic.go:356] "Generic (PLEG): container finished" podID="37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc" containerID="8192c9d6f6c8bf662136e444d5a3385044f95a22fb2f0751963a6f2941899590" exitCode=0 Jul 07 16:15:34.880362 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.880262 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" Jul 07 16:15:34.880362 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.880302 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" event={"ID":"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc","Type":"ContainerDied","Data":"8192c9d6f6c8bf662136e444d5a3385044f95a22fb2f0751963a6f2941899590"} Jul 07 16:15:34.880362 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.880360 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb" event={"ID":"37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc","Type":"ContainerDied","Data":"644625f89e8ec77cce88c292b5d8448bd8b3fb28721d2f79985fffaf2bb32f61"} Jul 07 16:15:34.880468 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.880383 2575 scope.go:117] "RemoveContainer" containerID="8192c9d6f6c8bf662136e444d5a3385044f95a22fb2f0751963a6f2941899590" Jul 07 16:15:34.889158 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.889141 2575 scope.go:117] "RemoveContainer" containerID="3c4398a7cd7024c180226fa4962ec83ea7e73cc0f579dfecab1f657810887ba2" Jul 07 16:15:34.896805 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.896789 2575 scope.go:117] "RemoveContainer" containerID="8192c9d6f6c8bf662136e444d5a3385044f95a22fb2f0751963a6f2941899590" Jul 07 16:15:34.897037 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:15:34.897019 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8192c9d6f6c8bf662136e444d5a3385044f95a22fb2f0751963a6f2941899590\": container with ID starting with 8192c9d6f6c8bf662136e444d5a3385044f95a22fb2f0751963a6f2941899590 not found: ID does not exist" containerID="8192c9d6f6c8bf662136e444d5a3385044f95a22fb2f0751963a6f2941899590" Jul 07 16:15:34.897088 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.897045 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8192c9d6f6c8bf662136e444d5a3385044f95a22fb2f0751963a6f2941899590"} err="failed to get container status \"8192c9d6f6c8bf662136e444d5a3385044f95a22fb2f0751963a6f2941899590\": rpc error: code = NotFound desc = could not find container \"8192c9d6f6c8bf662136e444d5a3385044f95a22fb2f0751963a6f2941899590\": container with ID starting with 8192c9d6f6c8bf662136e444d5a3385044f95a22fb2f0751963a6f2941899590 not found: ID does not exist" Jul 07 16:15:34.897088 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.897061 2575 scope.go:117] "RemoveContainer" containerID="3c4398a7cd7024c180226fa4962ec83ea7e73cc0f579dfecab1f657810887ba2" Jul 07 16:15:34.897288 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:15:34.897271 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c4398a7cd7024c180226fa4962ec83ea7e73cc0f579dfecab1f657810887ba2\": container with ID starting with 3c4398a7cd7024c180226fa4962ec83ea7e73cc0f579dfecab1f657810887ba2 not found: ID does not exist" containerID="3c4398a7cd7024c180226fa4962ec83ea7e73cc0f579dfecab1f657810887ba2" Jul 07 16:15:34.897325 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.897295 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c4398a7cd7024c180226fa4962ec83ea7e73cc0f579dfecab1f657810887ba2"} err="failed to get container status \"3c4398a7cd7024c180226fa4962ec83ea7e73cc0f579dfecab1f657810887ba2\": rpc error: code = NotFound desc = could not find container \"3c4398a7cd7024c180226fa4962ec83ea7e73cc0f579dfecab1f657810887ba2\": container with ID starting with 3c4398a7cd7024c180226fa4962ec83ea7e73cc0f579dfecab1f657810887ba2 not found: ID does not exist" Jul 07 16:15:34.905637 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.905613 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb"] Jul 07 16:15:34.909839 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:34.909818 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-4c489a-openshift-default-85744ccc4f-f5wxb"] Jul 07 16:15:35.436209 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:35.436175 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc" path="/var/lib/kubelet/pods/37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc/volumes" Jul 07 16:15:37.218815 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:37.218779 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246"] Jul 07 16:15:37.219233 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:37.219034 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" podUID="769bd66f-e164-483e-88b5-d2a136af3f11" containerName="istio-proxy" containerID="cri-o://275dfb595ccdb3d3043f3740c8e7d1231ea5b3a031aa3cea711b73dde2851d98" gracePeriod=30 Jul 07 16:15:38.108006 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.107970 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k"] Jul 07 16:15:38.108429 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.108409 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4e0ac2f-3992-4903-934a-616f68d11d99" containerName="maas-api" Jul 07 16:15:38.108525 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.108431 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4e0ac2f-3992-4903-934a-616f68d11d99" containerName="maas-api" Jul 07 16:15:38.108525 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.108457 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc" containerName="istio-proxy" Jul 07 16:15:38.108525 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.108466 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc" containerName="istio-proxy" Jul 07 16:15:38.108675 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.108575 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc" containerName="istio-proxy" Jul 07 16:15:38.108675 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.108590 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4e0ac2f-3992-4903-934a-616f68d11d99" containerName="maas-api" Jul 07 16:15:38.111558 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.111534 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.114971 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.114951 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-9ac19f-openshift-default-dockercfg-ctkn7\"" Jul 07 16:15:38.125417 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.125392 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k"] Jul 07 16:15:38.167026 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.167000 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istio-envoy\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.167140 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.167040 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istiod-ca-cert\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.167140 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.167063 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istio-podinfo\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.167140 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.167103 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istio-token\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.167258 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.167197 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m94t5\" (UniqueName: \"kubernetes.io/projected/c3728dfe-514c-4b21-a1f8-757825bb4a5e-kube-api-access-m94t5\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.167258 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.167235 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istio-data\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.167327 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.167259 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-workload-certs\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.167327 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.167312 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-credential-socket\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.167391 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.167329 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-workload-socket\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.268103 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.268067 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-workload-certs\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.268534 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.268112 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-credential-socket\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.268534 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.268225 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-workload-socket\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.268534 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.268293 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istio-envoy\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.268534 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.268338 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istiod-ca-cert\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.268534 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.268371 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istio-podinfo\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.268534 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.268403 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istio-token\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.268534 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.268427 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-workload-certs\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.268534 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.268444 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m94t5\" (UniqueName: \"kubernetes.io/projected/c3728dfe-514c-4b21-a1f8-757825bb4a5e-kube-api-access-m94t5\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.268534 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.268483 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-credential-socket\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.268534 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.268508 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istio-data\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.269088 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.268566 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-workload-socket\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.269088 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.268804 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istio-data\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.269088 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.268986 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istiod-ca-cert\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.270643 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.270616 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istio-envoy\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.270788 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.270768 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istio-podinfo\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.277464 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.277442 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istio-token\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.277684 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.277654 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m94t5\" (UniqueName: \"kubernetes.io/projected/c3728dfe-514c-4b21-a1f8-757825bb4a5e-kube-api-access-m94t5\") pod \"e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.421750 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.421667 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:38.558231 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.558059 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k"] Jul 07 16:15:38.560504 ip-10-0-131-221 kubenswrapper[2575]: W0707 16:15:38.560474 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3728dfe_514c_4b21_a1f8_757825bb4a5e.slice/crio-372307b5cb1bd1dcbf3579befe407d77d7b91affe8765ef8c635d52ae58c5b85 WatchSource:0}: Error finding container 372307b5cb1bd1dcbf3579befe407d77d7b91affe8765ef8c635d52ae58c5b85: Status 404 returned error can't find the container with id 372307b5cb1bd1dcbf3579befe407d77d7b91affe8765ef8c635d52ae58c5b85 Jul 07 16:15:38.562965 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.562930 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:15:38.563057 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.563003 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:15:38.563057 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.563035 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:15:38.902957 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.902913 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" event={"ID":"c3728dfe-514c-4b21-a1f8-757825bb4a5e","Type":"ContainerStarted","Data":"ef98834a075b9210b326b22bed58f110dbcdb69378dd4e3cef2b5b1ec43231a5"} Jul 07 16:15:38.903138 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.902978 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" event={"ID":"c3728dfe-514c-4b21-a1f8-757825bb4a5e","Type":"ContainerStarted","Data":"372307b5cb1bd1dcbf3579befe407d77d7b91affe8765ef8c635d52ae58c5b85"} Jul 07 16:15:38.939283 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:38.939233 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" podStartSLOduration=0.939215842 podStartE2EDuration="939.215842ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:15:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:15:38.936514317 +0000 UTC m=+2026.031795387" watchObservedRunningTime="2026-07-07 16:15:38.939215842 +0000 UTC m=+2026.034496920" Jul 07 16:15:39.422529 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:39.422496 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:39.427061 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:39.427037 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:39.908392 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:39.908356 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:39.909221 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:39.909197 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:40.967983 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:40.967945 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq"] Jul 07 16:15:40.968518 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:40.968502 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc" containerName="istio-proxy" Jul 07 16:15:40.968566 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:40.968521 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc" containerName="istio-proxy" Jul 07 16:15:40.968645 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:40.968635 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="37abdaf0-40d4-48e8-9fe8-eeb9faf73cbc" containerName="istio-proxy" Jul 07 16:15:40.971065 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:40.971048 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq" Jul 07 16:15:40.973666 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:40.973639 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-9ac19f-serving-cert\"" Jul 07 16:15:40.973785 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:40.973732 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-5vqnq\"" Jul 07 16:15:40.983129 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:40.983106 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq"] Jul 07 16:15:41.094521 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:41.094482 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t4z5\" (UniqueName: \"kubernetes.io/projected/038e063a-961b-418d-9155-3b094ce011f0-kube-api-access-4t4z5\") pod \"maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq\" (UID: \"038e063a-961b-418d-9155-3b094ce011f0\") " pod="opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq" Jul 07 16:15:41.094702 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:41.094611 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/038e063a-961b-418d-9155-3b094ce011f0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq\" (UID: \"038e063a-961b-418d-9155-3b094ce011f0\") " pod="opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq" Jul 07 16:15:41.094702 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:41.094636 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/038e063a-961b-418d-9155-3b094ce011f0-maas-api-tls\") pod \"maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq\" (UID: \"038e063a-961b-418d-9155-3b094ce011f0\") " pod="opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq" Jul 07 16:15:41.195300 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:41.195269 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/038e063a-961b-418d-9155-3b094ce011f0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq\" (UID: \"038e063a-961b-418d-9155-3b094ce011f0\") " pod="opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq" Jul 07 16:15:41.195300 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:41.195306 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/038e063a-961b-418d-9155-3b094ce011f0-maas-api-tls\") pod \"maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq\" (UID: \"038e063a-961b-418d-9155-3b094ce011f0\") " pod="opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq" Jul 07 16:15:41.195517 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:41.195364 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t4z5\" (UniqueName: \"kubernetes.io/projected/038e063a-961b-418d-9155-3b094ce011f0-kube-api-access-4t4z5\") pod \"maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq\" (UID: \"038e063a-961b-418d-9155-3b094ce011f0\") " pod="opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq" Jul 07 16:15:41.195924 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:41.195901 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/038e063a-961b-418d-9155-3b094ce011f0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq\" (UID: \"038e063a-961b-418d-9155-3b094ce011f0\") " pod="opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq" Jul 07 16:15:41.197762 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:41.197740 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/038e063a-961b-418d-9155-3b094ce011f0-maas-api-tls\") pod \"maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq\" (UID: \"038e063a-961b-418d-9155-3b094ce011f0\") " pod="opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq" Jul 07 16:15:41.208041 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:41.208020 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t4z5\" (UniqueName: \"kubernetes.io/projected/038e063a-961b-418d-9155-3b094ce011f0-kube-api-access-4t4z5\") pod \"maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq\" (UID: \"038e063a-961b-418d-9155-3b094ce011f0\") " pod="opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq" Jul 07 16:15:41.281273 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:41.281205 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq" Jul 07 16:15:41.621574 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:41.621548 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq"] Jul 07 16:15:41.622236 ip-10-0-131-221 kubenswrapper[2575]: W0707 16:15:41.622209 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod038e063a_961b_418d_9155_3b094ce011f0.slice/crio-185f377721daddeae75b1867b1d414dddd9c5ad05aa611cadef535404b27de28 WatchSource:0}: Error finding container 185f377721daddeae75b1867b1d414dddd9c5ad05aa611cadef535404b27de28: Status 404 returned error can't find the container with id 185f377721daddeae75b1867b1d414dddd9c5ad05aa611cadef535404b27de28 Jul 07 16:15:41.926524 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:41.926441 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq" event={"ID":"038e063a-961b-418d-9155-3b094ce011f0","Type":"ContainerStarted","Data":"185f377721daddeae75b1867b1d414dddd9c5ad05aa611cadef535404b27de28"} Jul 07 16:15:42.480015 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.479987 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:42.611579 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.611544 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-workload-certs\") pod \"769bd66f-e164-483e-88b5-d2a136af3f11\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " Jul 07 16:15:42.611747 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.611601 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-credential-socket\") pod \"769bd66f-e164-483e-88b5-d2a136af3f11\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " Jul 07 16:15:42.611747 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.611623 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-istio-envoy\") pod \"769bd66f-e164-483e-88b5-d2a136af3f11\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " Jul 07 16:15:42.611747 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.611687 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-istio-data\") pod \"769bd66f-e164-483e-88b5-d2a136af3f11\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " Jul 07 16:15:42.611747 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.611722 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/769bd66f-e164-483e-88b5-d2a136af3f11-istio-podinfo\") pod \"769bd66f-e164-483e-88b5-d2a136af3f11\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " Jul 07 16:15:42.611958 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.611756 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mg59p\" (UniqueName: \"kubernetes.io/projected/769bd66f-e164-483e-88b5-d2a136af3f11-kube-api-access-mg59p\") pod \"769bd66f-e164-483e-88b5-d2a136af3f11\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " Jul 07 16:15:42.611958 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.611778 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/769bd66f-e164-483e-88b5-d2a136af3f11-istiod-ca-cert\") pod \"769bd66f-e164-483e-88b5-d2a136af3f11\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " Jul 07 16:15:42.611958 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.611838 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-workload-socket\") pod \"769bd66f-e164-483e-88b5-d2a136af3f11\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " Jul 07 16:15:42.611958 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.611874 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/769bd66f-e164-483e-88b5-d2a136af3f11-istio-token\") pod \"769bd66f-e164-483e-88b5-d2a136af3f11\" (UID: \"769bd66f-e164-483e-88b5-d2a136af3f11\") " Jul 07 16:15:42.611958 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.611834 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "769bd66f-e164-483e-88b5-d2a136af3f11" (UID: "769bd66f-e164-483e-88b5-d2a136af3f11"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:15:42.611958 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.611942 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "769bd66f-e164-483e-88b5-d2a136af3f11" (UID: "769bd66f-e164-483e-88b5-d2a136af3f11"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:15:42.612274 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.612177 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-workload-certs\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:42.612274 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.612190 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/769bd66f-e164-483e-88b5-d2a136af3f11-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "769bd66f-e164-483e-88b5-d2a136af3f11" (UID: "769bd66f-e164-483e-88b5-d2a136af3f11"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:15:42.612274 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.612196 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-credential-socket\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:42.612274 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.612209 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "769bd66f-e164-483e-88b5-d2a136af3f11" (UID: "769bd66f-e164-483e-88b5-d2a136af3f11"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:15:42.612796 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.612770 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-istio-data" (OuterVolumeSpecName: "istio-data") pod "769bd66f-e164-483e-88b5-d2a136af3f11" (UID: "769bd66f-e164-483e-88b5-d2a136af3f11"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:15:42.613891 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.613843 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "769bd66f-e164-483e-88b5-d2a136af3f11" (UID: "769bd66f-e164-483e-88b5-d2a136af3f11"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:15:42.614040 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.613901 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/769bd66f-e164-483e-88b5-d2a136af3f11-kube-api-access-mg59p" (OuterVolumeSpecName: "kube-api-access-mg59p") pod "769bd66f-e164-483e-88b5-d2a136af3f11" (UID: "769bd66f-e164-483e-88b5-d2a136af3f11"). InnerVolumeSpecName "kube-api-access-mg59p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:15:42.614109 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.614091 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/769bd66f-e164-483e-88b5-d2a136af3f11-istio-token" (OuterVolumeSpecName: "istio-token") pod "769bd66f-e164-483e-88b5-d2a136af3f11" (UID: "769bd66f-e164-483e-88b5-d2a136af3f11"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:15:42.614147 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.614133 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/769bd66f-e164-483e-88b5-d2a136af3f11-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "769bd66f-e164-483e-88b5-d2a136af3f11" (UID: "769bd66f-e164-483e-88b5-d2a136af3f11"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 16:15:42.713092 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.713059 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/769bd66f-e164-483e-88b5-d2a136af3f11-istio-podinfo\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:42.713092 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.713087 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mg59p\" (UniqueName: \"kubernetes.io/projected/769bd66f-e164-483e-88b5-d2a136af3f11-kube-api-access-mg59p\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:42.713276 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.713105 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/769bd66f-e164-483e-88b5-d2a136af3f11-istiod-ca-cert\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:42.713276 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.713116 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-workload-socket\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:42.713276 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.713128 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/769bd66f-e164-483e-88b5-d2a136af3f11-istio-token\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:42.713276 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.713139 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-istio-envoy\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:42.713276 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.713149 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/769bd66f-e164-483e-88b5-d2a136af3f11-istio-data\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:15:42.932902 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.932795 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq" event={"ID":"038e063a-961b-418d-9155-3b094ce011f0","Type":"ContainerStarted","Data":"4522c48ba30c37ce734181365616e78979a163c16021d5b67fe748dc44fd1c6d"} Jul 07 16:15:42.932902 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.932898 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq" Jul 07 16:15:42.934377 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.934348 2575 generic.go:356] "Generic (PLEG): container finished" podID="769bd66f-e164-483e-88b5-d2a136af3f11" containerID="275dfb595ccdb3d3043f3740c8e7d1231ea5b3a031aa3cea711b73dde2851d98" exitCode=0 Jul 07 16:15:42.934523 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.934394 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" Jul 07 16:15:42.934523 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.934424 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" event={"ID":"769bd66f-e164-483e-88b5-d2a136af3f11","Type":"ContainerDied","Data":"275dfb595ccdb3d3043f3740c8e7d1231ea5b3a031aa3cea711b73dde2851d98"} Jul 07 16:15:42.934523 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.934475 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246" event={"ID":"769bd66f-e164-483e-88b5-d2a136af3f11","Type":"ContainerDied","Data":"8048be5497244cdb257cfa03b9cc4d7c0135e6a32ef32a70cedee4214a816c4f"} Jul 07 16:15:42.934523 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.934500 2575 scope.go:117] "RemoveContainer" containerID="275dfb595ccdb3d3043f3740c8e7d1231ea5b3a031aa3cea711b73dde2851d98" Jul 07 16:15:42.943170 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.943146 2575 scope.go:117] "RemoveContainer" containerID="b7b7f0f3d39b8bf5a811158bf7dfc902c50fbd882b8e7c8e9ef739ce5292a882" Jul 07 16:15:42.951236 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.951217 2575 scope.go:117] "RemoveContainer" containerID="275dfb595ccdb3d3043f3740c8e7d1231ea5b3a031aa3cea711b73dde2851d98" Jul 07 16:15:42.951483 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:15:42.951464 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"275dfb595ccdb3d3043f3740c8e7d1231ea5b3a031aa3cea711b73dde2851d98\": container with ID starting with 275dfb595ccdb3d3043f3740c8e7d1231ea5b3a031aa3cea711b73dde2851d98 not found: ID does not exist" containerID="275dfb595ccdb3d3043f3740c8e7d1231ea5b3a031aa3cea711b73dde2851d98" Jul 07 16:15:42.951535 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.951492 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"275dfb595ccdb3d3043f3740c8e7d1231ea5b3a031aa3cea711b73dde2851d98"} err="failed to get container status \"275dfb595ccdb3d3043f3740c8e7d1231ea5b3a031aa3cea711b73dde2851d98\": rpc error: code = NotFound desc = could not find container \"275dfb595ccdb3d3043f3740c8e7d1231ea5b3a031aa3cea711b73dde2851d98\": container with ID starting with 275dfb595ccdb3d3043f3740c8e7d1231ea5b3a031aa3cea711b73dde2851d98 not found: ID does not exist" Jul 07 16:15:42.951535 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.951511 2575 scope.go:117] "RemoveContainer" containerID="b7b7f0f3d39b8bf5a811158bf7dfc902c50fbd882b8e7c8e9ef739ce5292a882" Jul 07 16:15:42.951752 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:15:42.951734 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7b7f0f3d39b8bf5a811158bf7dfc902c50fbd882b8e7c8e9ef739ce5292a882\": container with ID starting with b7b7f0f3d39b8bf5a811158bf7dfc902c50fbd882b8e7c8e9ef739ce5292a882 not found: ID does not exist" containerID="b7b7f0f3d39b8bf5a811158bf7dfc902c50fbd882b8e7c8e9ef739ce5292a882" Jul 07 16:15:42.951812 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.951761 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7b7f0f3d39b8bf5a811158bf7dfc902c50fbd882b8e7c8e9ef739ce5292a882"} err="failed to get container status \"b7b7f0f3d39b8bf5a811158bf7dfc902c50fbd882b8e7c8e9ef739ce5292a882\": rpc error: code = NotFound desc = could not find container \"b7b7f0f3d39b8bf5a811158bf7dfc902c50fbd882b8e7c8e9ef739ce5292a882\": container with ID starting with b7b7f0f3d39b8bf5a811158bf7dfc902c50fbd882b8e7c8e9ef739ce5292a882 not found: ID does not exist" Jul 07 16:15:42.956269 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.956090 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq" podStartSLOduration=2.570966409 podStartE2EDuration="2.956075266s" totalImagesPullingTime="385.108857ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:15:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:15:42.954283308 +0000 UTC m=+2030.049564386" watchObservedRunningTime="2026-07-07 16:15:42.956075266 +0000 UTC m=+2030.051356347" Jul 07 16:15:42.977140 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.977112 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246"] Jul 07 16:15:42.983443 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:42.983423 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-b155c0-openshift-default-57f44d76c7-xs246"] Jul 07 16:15:43.436077 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:43.436046 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="769bd66f-e164-483e-88b5-d2a136af3f11" path="/var/lib/kubelet/pods/769bd66f-e164-483e-88b5-d2a136af3f11/volumes" Jul 07 16:15:46.074045 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.074011 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9"] Jul 07 16:15:46.074465 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.074448 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="769bd66f-e164-483e-88b5-d2a136af3f11" containerName="istio-proxy" Jul 07 16:15:46.074512 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.074468 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="769bd66f-e164-483e-88b5-d2a136af3f11" containerName="istio-proxy" Jul 07 16:15:46.074512 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.074480 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="769bd66f-e164-483e-88b5-d2a136af3f11" containerName="istio-proxy" Jul 07 16:15:46.074512 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.074485 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="769bd66f-e164-483e-88b5-d2a136af3f11" containerName="istio-proxy" Jul 07 16:15:46.074606 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.074535 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="769bd66f-e164-483e-88b5-d2a136af3f11" containerName="istio-proxy" Jul 07 16:15:46.074606 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.074542 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="769bd66f-e164-483e-88b5-d2a136af3f11" containerName="istio-proxy" Jul 07 16:15:46.077710 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.077691 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.081082 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.081058 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-132358-openshift-default-dockercfg-9ttnc\"" Jul 07 16:15:46.096422 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.096392 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9"] Jul 07 16:15:46.249100 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.249066 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-workload-certs\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.249100 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.249103 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istio-data\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.249367 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.249122 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxkpc\" (UniqueName: \"kubernetes.io/projected/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-kube-api-access-jxkpc\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.249367 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.249223 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istio-podinfo\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.249367 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.249270 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istio-envoy\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.249367 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.249327 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-credential-socket\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.249367 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.249360 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istio-token\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.249595 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.249401 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-workload-socket\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.249595 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.249432 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istiod-ca-cert\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.350717 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.350684 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-workload-certs\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.350929 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.350730 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istio-data\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.350929 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.350748 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxkpc\" (UniqueName: \"kubernetes.io/projected/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-kube-api-access-jxkpc\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.350929 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.350777 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istio-podinfo\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.350929 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.350802 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istio-envoy\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.350929 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.350841 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-credential-socket\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.350929 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.350857 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istio-token\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.351268 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.350972 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-workload-socket\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.351268 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.350995 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istiod-ca-cert\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.351268 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.351260 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istio-data\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.351427 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.351332 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-workload-certs\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.351427 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.351362 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-credential-socket\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.351511 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.351434 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-workload-socket\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.351749 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.351730 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istiod-ca-cert\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.353241 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.353214 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istio-envoy\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.353336 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.353324 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istio-podinfo\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.361421 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.361398 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxkpc\" (UniqueName: \"kubernetes.io/projected/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-kube-api-access-jxkpc\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.361569 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.361551 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istio-token\") pod \"e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.388314 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.388280 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:46.533347 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.533319 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9"] Jul 07 16:15:46.535289 ip-10-0-131-221 kubenswrapper[2575]: W0707 16:15:46.535259 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddda05f2e_9d64_4f4d_9d05_d5a80b576d2e.slice/crio-60f1ad499af95c20f2c4c1e01e2fe0a7cf2275c5742a98991578cbc9c87af2f5 WatchSource:0}: Error finding container 60f1ad499af95c20f2c4c1e01e2fe0a7cf2275c5742a98991578cbc9c87af2f5: Status 404 returned error can't find the container with id 60f1ad499af95c20f2c4c1e01e2fe0a7cf2275c5742a98991578cbc9c87af2f5 Jul 07 16:15:46.537648 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.537613 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:15:46.537735 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.537687 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:15:46.537735 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.537718 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:15:46.956689 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.956657 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" event={"ID":"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e","Type":"ContainerStarted","Data":"5025ca7f513bb0a2a67cd5f18369d0eddad95d9700d992f0d1519c3a0fcca014"} Jul 07 16:15:46.956903 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.956711 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" event={"ID":"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e","Type":"ContainerStarted","Data":"60f1ad499af95c20f2c4c1e01e2fe0a7cf2275c5742a98991578cbc9c87af2f5"} Jul 07 16:15:46.989897 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:46.989828 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" podStartSLOduration=0.989811778 podStartE2EDuration="989.811778ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:15:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:15:46.987279807 +0000 UTC m=+2034.082560888" watchObservedRunningTime="2026-07-07 16:15:46.989811778 +0000 UTC m=+2034.085092857" Jul 07 16:15:47.388695 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:47.388661 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:47.393573 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:47.393551 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:47.964846 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:47.964807 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:47.965909 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:47.965885 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:15:48.952320 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:48.952281 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq" Jul 07 16:15:55.039713 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:55.039165 2575 patch_prober.go:28] interesting pod/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": read tcp 10.132.0.2:37672->10.132.0.47:15021: read: connection reset by peer" start-of-body= Jul 07 16:15:55.039713 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:55.039222 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" podUID="c3728dfe-514c-4b21-a1f8-757825bb4a5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": read tcp 10.132.0.2:37672->10.132.0.47:15021: read: connection reset by peer" Jul 07 16:15:56.008313 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:56.008278 2575 generic.go:356] "Generic (PLEG): container finished" podID="c3728dfe-514c-4b21-a1f8-757825bb4a5e" containerID="ef98834a075b9210b326b22bed58f110dbcdb69378dd4e3cef2b5b1ec43231a5" exitCode=0 Jul 07 16:15:56.008604 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:56.008349 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" event={"ID":"c3728dfe-514c-4b21-a1f8-757825bb4a5e","Type":"ContainerDied","Data":"ef98834a075b9210b326b22bed58f110dbcdb69378dd4e3cef2b5b1ec43231a5"} Jul 07 16:15:56.008799 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:56.008783 2575 scope.go:117] "RemoveContainer" containerID="ef98834a075b9210b326b22bed58f110dbcdb69378dd4e3cef2b5b1ec43231a5" Jul 07 16:15:56.009987 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:56.009956 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:15:56.010061 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:56.010040 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:15:56.010104 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:56.010077 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:15:57.015697 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:57.015663 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" event={"ID":"c3728dfe-514c-4b21-a1f8-757825bb4a5e","Type":"ContainerStarted","Data":"1a7eda745ae26d58600bd8cb8b2706b1da3245290fe420b3d848797688cb9341"} Jul 07 16:15:57.422622 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:57.422582 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:15:57.422940 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:57.422914 2575 patch_prober.go:28] interesting pod/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 07 16:15:57.423012 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:57.422977 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" podUID="c3728dfe-514c-4b21-a1f8-757825bb4a5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 07 16:15:58.422466 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:58.422422 2575 patch_prober.go:28] interesting pod/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 07 16:15:58.422883 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:58.422499 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" podUID="c3728dfe-514c-4b21-a1f8-757825bb4a5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 07 16:15:59.423042 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:59.423006 2575 patch_prober.go:28] interesting pod/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 07 16:15:59.423597 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:15:59.423064 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" podUID="c3728dfe-514c-4b21-a1f8-757825bb4a5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 07 16:16:00.422980 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:00.422857 2575 patch_prober.go:28] interesting pod/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 07 16:16:00.422980 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:00.422958 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" podUID="c3728dfe-514c-4b21-a1f8-757825bb4a5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 07 16:16:01.037442 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:01.037405 2575 generic.go:356] "Generic (PLEG): container finished" podID="dda05f2e-9d64-4f4d-9d05-d5a80b576d2e" containerID="5025ca7f513bb0a2a67cd5f18369d0eddad95d9700d992f0d1519c3a0fcca014" exitCode=0 Jul 07 16:16:01.037668 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:01.037472 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" event={"ID":"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e","Type":"ContainerDied","Data":"5025ca7f513bb0a2a67cd5f18369d0eddad95d9700d992f0d1519c3a0fcca014"} Jul 07 16:16:01.037942 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:01.037924 2575 scope.go:117] "RemoveContainer" containerID="5025ca7f513bb0a2a67cd5f18369d0eddad95d9700d992f0d1519c3a0fcca014" Jul 07 16:16:01.039037 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:01.038995 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:16:01.039268 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:01.039245 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:16:01.039422 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:01.039404 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:16:01.426337 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:01.426306 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:16:01.426769 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:01.426606 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:16:01.427256 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:01.427238 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:16:02.044390 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:02.044352 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" event={"ID":"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e","Type":"ContainerStarted","Data":"d3c41207bebccb5d8aa6d23fb0af9791beb2a7a10c18eb0c3ad0a13430e7cad8"} Jul 07 16:16:02.389495 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:02.389458 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:16:02.390638 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:02.390615 2575 patch_prober.go:28] interesting pod/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 07 16:16:02.390749 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:02.390664 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" podUID="dda05f2e-9d64-4f4d-9d05-d5a80b576d2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 07 16:16:02.965481 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:02.965439 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:16:03.389521 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:03.389483 2575 patch_prober.go:28] interesting pod/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 07 16:16:03.389714 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:03.389558 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" podUID="dda05f2e-9d64-4f4d-9d05-d5a80b576d2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 07 16:16:04.389601 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:04.389570 2575 patch_prober.go:28] interesting pod/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 07 16:16:04.390010 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:04.389635 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" podUID="dda05f2e-9d64-4f4d-9d05-d5a80b576d2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 07 16:16:05.389641 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:05.389600 2575 patch_prober.go:28] interesting pod/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 07 16:16:05.390097 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:05.389679 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" podUID="dda05f2e-9d64-4f4d-9d05-d5a80b576d2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 07 16:16:06.388915 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:06.388852 2575 patch_prober.go:28] interesting pod/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 07 16:16:06.389103 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:06.388952 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" podUID="dda05f2e-9d64-4f4d-9d05-d5a80b576d2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 07 16:16:07.392880 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:07.392843 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:16:07.394099 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:07.394078 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:16:53.505288 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:53.505259 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nfh8b_162f95df-a98f-49a6-a91e-689a8e22be46/console-operator/1.log" Jul 07 16:16:53.507655 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:53.507632 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nfh8b_162f95df-a98f-49a6-a91e-689a8e22be46/console-operator/1.log" Jul 07 16:16:53.514430 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:53.514410 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4d86s_fbf78c29-89f3-4aad-82a7-2e8f99b6356e/ovn-acl-logging/0.log" Jul 07 16:16:53.523435 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:16:53.523415 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4d86s_fbf78c29-89f3-4aad-82a7-2e8f99b6356e/ovn-acl-logging/0.log" Jul 07 16:21:13.561987 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:13.561952 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq"] Jul 07 16:21:13.563582 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:13.562201 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq" podUID="038e063a-961b-418d-9155-3b094ce011f0" containerName="maas-api" containerID="cri-o://4522c48ba30c37ce734181365616e78979a163c16021d5b67fe748dc44fd1c6d" gracePeriod=30 Jul 07 16:21:14.007298 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:14.007265 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq" Jul 07 16:21:14.178276 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:14.178187 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4t4z5\" (UniqueName: \"kubernetes.io/projected/038e063a-961b-418d-9155-3b094ce011f0-kube-api-access-4t4z5\") pod \"038e063a-961b-418d-9155-3b094ce011f0\" (UID: \"038e063a-961b-418d-9155-3b094ce011f0\") " Jul 07 16:21:14.178276 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:14.178255 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/038e063a-961b-418d-9155-3b094ce011f0-openshift-service-ca\") pod \"038e063a-961b-418d-9155-3b094ce011f0\" (UID: \"038e063a-961b-418d-9155-3b094ce011f0\") " Jul 07 16:21:14.178535 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:14.178287 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/038e063a-961b-418d-9155-3b094ce011f0-maas-api-tls\") pod \"038e063a-961b-418d-9155-3b094ce011f0\" (UID: \"038e063a-961b-418d-9155-3b094ce011f0\") " Jul 07 16:21:14.178682 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:14.178656 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/038e063a-961b-418d-9155-3b094ce011f0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "038e063a-961b-418d-9155-3b094ce011f0" (UID: "038e063a-961b-418d-9155-3b094ce011f0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:21:14.180436 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:14.180412 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/038e063a-961b-418d-9155-3b094ce011f0-kube-api-access-4t4z5" (OuterVolumeSpecName: "kube-api-access-4t4z5") pod "038e063a-961b-418d-9155-3b094ce011f0" (UID: "038e063a-961b-418d-9155-3b094ce011f0"). InnerVolumeSpecName "kube-api-access-4t4z5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:21:14.180529 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:14.180437 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/038e063a-961b-418d-9155-3b094ce011f0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "038e063a-961b-418d-9155-3b094ce011f0" (UID: "038e063a-961b-418d-9155-3b094ce011f0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:21:14.279267 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:14.279230 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/038e063a-961b-418d-9155-3b094ce011f0-openshift-service-ca\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:21:14.279267 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:14.279262 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/038e063a-961b-418d-9155-3b094ce011f0-maas-api-tls\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:21:14.279267 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:14.279275 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4t4z5\" (UniqueName: \"kubernetes.io/projected/038e063a-961b-418d-9155-3b094ce011f0-kube-api-access-4t4z5\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:21:14.585509 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:14.585407 2575 generic.go:356] "Generic (PLEG): container finished" podID="038e063a-961b-418d-9155-3b094ce011f0" containerID="4522c48ba30c37ce734181365616e78979a163c16021d5b67fe748dc44fd1c6d" exitCode=0 Jul 07 16:21:14.585509 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:14.585453 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq" Jul 07 16:21:14.585509 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:14.585479 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq" event={"ID":"038e063a-961b-418d-9155-3b094ce011f0","Type":"ContainerDied","Data":"4522c48ba30c37ce734181365616e78979a163c16021d5b67fe748dc44fd1c6d"} Jul 07 16:21:14.586101 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:14.585551 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq" event={"ID":"038e063a-961b-418d-9155-3b094ce011f0","Type":"ContainerDied","Data":"185f377721daddeae75b1867b1d414dddd9c5ad05aa611cadef535404b27de28"} Jul 07 16:21:14.586101 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:14.585581 2575 scope.go:117] "RemoveContainer" containerID="4522c48ba30c37ce734181365616e78979a163c16021d5b67fe748dc44fd1c6d" Jul 07 16:21:14.596930 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:14.596905 2575 scope.go:117] "RemoveContainer" containerID="4522c48ba30c37ce734181365616e78979a163c16021d5b67fe748dc44fd1c6d" Jul 07 16:21:14.597235 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:21:14.597217 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4522c48ba30c37ce734181365616e78979a163c16021d5b67fe748dc44fd1c6d\": container with ID starting with 4522c48ba30c37ce734181365616e78979a163c16021d5b67fe748dc44fd1c6d not found: ID does not exist" containerID="4522c48ba30c37ce734181365616e78979a163c16021d5b67fe748dc44fd1c6d" Jul 07 16:21:14.597283 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:14.597247 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4522c48ba30c37ce734181365616e78979a163c16021d5b67fe748dc44fd1c6d"} err="failed to get container status \"4522c48ba30c37ce734181365616e78979a163c16021d5b67fe748dc44fd1c6d\": rpc error: code = NotFound desc = could not find container \"4522c48ba30c37ce734181365616e78979a163c16021d5b67fe748dc44fd1c6d\": container with ID starting with 4522c48ba30c37ce734181365616e78979a163c16021d5b67fe748dc44fd1c6d not found: ID does not exist" Jul 07 16:21:14.610984 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:14.610955 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq"] Jul 07 16:21:14.618187 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:14.618165 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq"] Jul 07 16:21:15.436402 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:15.436371 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="038e063a-961b-418d-9155-3b094ce011f0" path="/var/lib/kubelet/pods/038e063a-961b-418d-9155-3b094ce011f0/volumes" Jul 07 16:21:16.942250 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:16.942209 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-shared-a-9ac19f-85949f745c-8mnvq" podUID="038e063a-961b-418d-9155-3b094ce011f0" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.48:8443/health\": context deadline exceeded" Jul 07 16:21:53.543881 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:53.543830 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nfh8b_162f95df-a98f-49a6-a91e-689a8e22be46/console-operator/1.log" Jul 07 16:21:53.550936 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:53.550913 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4d86s_fbf78c29-89f3-4aad-82a7-2e8f99b6356e/ovn-acl-logging/0.log" Jul 07 16:21:53.551080 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:53.550998 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nfh8b_162f95df-a98f-49a6-a91e-689a8e22be46/console-operator/1.log" Jul 07 16:21:53.557773 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:53.557754 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4d86s_fbf78c29-89f3-4aad-82a7-2e8f99b6356e/ovn-acl-logging/0.log" Jul 07 16:21:56.643577 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:56.643539 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k"] Jul 07 16:21:56.644079 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:21:56.643801 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" podUID="c3728dfe-514c-4b21-a1f8-757825bb4a5e" containerName="istio-proxy" containerID="cri-o://1a7eda745ae26d58600bd8cb8b2706b1da3245290fe420b3d848797688cb9341" gracePeriod=30 Jul 07 16:22:01.429522 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.429476 2575 patch_prober.go:28] interesting pod/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 16:22:01.429960 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.429545 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" podUID="c3728dfe-514c-4b21-a1f8-757825bb4a5e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 16:22:01.816272 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.816241 2575 generic.go:356] "Generic (PLEG): container finished" podID="c3728dfe-514c-4b21-a1f8-757825bb4a5e" containerID="1a7eda745ae26d58600bd8cb8b2706b1da3245290fe420b3d848797688cb9341" exitCode=0 Jul 07 16:22:01.816415 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.816320 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" event={"ID":"c3728dfe-514c-4b21-a1f8-757825bb4a5e","Type":"ContainerDied","Data":"1a7eda745ae26d58600bd8cb8b2706b1da3245290fe420b3d848797688cb9341"} Jul 07 16:22:01.816415 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.816395 2575 scope.go:117] "RemoveContainer" containerID="ef98834a075b9210b326b22bed58f110dbcdb69378dd4e3cef2b5b1ec43231a5" Jul 07 16:22:01.898843 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.898816 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:22:01.937664 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.937630 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istio-podinfo\") pod \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " Jul 07 16:22:01.937846 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.937678 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-credential-socket\") pod \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " Jul 07 16:22:01.937846 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.937700 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istiod-ca-cert\") pod \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " Jul 07 16:22:01.937846 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.937725 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istio-data\") pod \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " Jul 07 16:22:01.938049 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.937901 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istio-token\") pod \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " Jul 07 16:22:01.938049 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.937926 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c3728dfe-514c-4b21-a1f8-757825bb4a5e" (UID: "c3728dfe-514c-4b21-a1f8-757825bb4a5e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:22:01.938049 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.937990 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m94t5\" (UniqueName: \"kubernetes.io/projected/c3728dfe-514c-4b21-a1f8-757825bb4a5e-kube-api-access-m94t5\") pod \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " Jul 07 16:22:01.938049 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.938023 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istio-envoy\") pod \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " Jul 07 16:22:01.938256 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.938057 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-workload-certs\") pod \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " Jul 07 16:22:01.938256 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.938096 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-workload-socket\") pod \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\" (UID: \"c3728dfe-514c-4b21-a1f8-757825bb4a5e\") " Jul 07 16:22:01.938256 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.938218 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c3728dfe-514c-4b21-a1f8-757825bb4a5e" (UID: "c3728dfe-514c-4b21-a1f8-757825bb4a5e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:22:01.938490 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.938467 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c3728dfe-514c-4b21-a1f8-757825bb4a5e" (UID: "c3728dfe-514c-4b21-a1f8-757825bb4a5e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:22:01.938569 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.938504 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istio-data" (OuterVolumeSpecName: "istio-data") pod "c3728dfe-514c-4b21-a1f8-757825bb4a5e" (UID: "c3728dfe-514c-4b21-a1f8-757825bb4a5e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:22:01.938609 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.938548 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-credential-socket\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:22:01.938609 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.938577 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c3728dfe-514c-4b21-a1f8-757825bb4a5e" (UID: "c3728dfe-514c-4b21-a1f8-757825bb4a5e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:22:01.938609 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.938592 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istiod-ca-cert\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:22:01.940209 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.940188 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istio-token" (OuterVolumeSpecName: "istio-token") pod "c3728dfe-514c-4b21-a1f8-757825bb4a5e" (UID: "c3728dfe-514c-4b21-a1f8-757825bb4a5e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:22:01.940341 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.940323 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c3728dfe-514c-4b21-a1f8-757825bb4a5e" (UID: "c3728dfe-514c-4b21-a1f8-757825bb4a5e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 16:22:01.940397 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.940387 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c3728dfe-514c-4b21-a1f8-757825bb4a5e" (UID: "c3728dfe-514c-4b21-a1f8-757825bb4a5e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:22:01.940679 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:01.940658 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3728dfe-514c-4b21-a1f8-757825bb4a5e-kube-api-access-m94t5" (OuterVolumeSpecName: "kube-api-access-m94t5") pod "c3728dfe-514c-4b21-a1f8-757825bb4a5e" (UID: "c3728dfe-514c-4b21-a1f8-757825bb4a5e"). InnerVolumeSpecName "kube-api-access-m94t5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:22:02.039421 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:02.039335 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istio-data\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:22:02.039421 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:02.039363 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istio-token\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:22:02.039421 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:02.039372 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m94t5\" (UniqueName: \"kubernetes.io/projected/c3728dfe-514c-4b21-a1f8-757825bb4a5e-kube-api-access-m94t5\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:22:02.039421 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:02.039381 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istio-envoy\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:22:02.039421 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:02.039390 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-workload-certs\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:22:02.039421 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:02.039397 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3728dfe-514c-4b21-a1f8-757825bb4a5e-workload-socket\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:22:02.039421 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:02.039404 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3728dfe-514c-4b21-a1f8-757825bb4a5e-istio-podinfo\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:22:02.822453 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:02.822420 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" Jul 07 16:22:02.822974 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:02.822415 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k" event={"ID":"c3728dfe-514c-4b21-a1f8-757825bb4a5e","Type":"ContainerDied","Data":"372307b5cb1bd1dcbf3579befe407d77d7b91affe8765ef8c635d52ae58c5b85"} Jul 07 16:22:02.822974 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:02.822560 2575 scope.go:117] "RemoveContainer" containerID="1a7eda745ae26d58600bd8cb8b2706b1da3245290fe420b3d848797688cb9341" Jul 07 16:22:02.846215 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:02.846186 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k"] Jul 07 16:22:02.851166 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:02.851142 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-9ac19f-openshift-default-7985b55b5f-ptx2k"] Jul 07 16:22:03.436669 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:03.436636 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3728dfe-514c-4b21-a1f8-757825bb4a5e" path="/var/lib/kubelet/pods/c3728dfe-514c-4b21-a1f8-757825bb4a5e/volumes" Jul 07 16:22:04.503649 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:04.503619 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9"] Jul 07 16:22:04.504084 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:04.503843 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" podUID="dda05f2e-9d64-4f4d-9d05-d5a80b576d2e" containerName="istio-proxy" containerID="cri-o://d3c41207bebccb5d8aa6d23fb0af9791beb2a7a10c18eb0c3ad0a13430e7cad8" gracePeriod=30 Jul 07 16:22:07.396123 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:07.396074 2575 patch_prober.go:28] interesting pod/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 16:22:07.396548 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:07.396149 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" podUID="dda05f2e-9d64-4f4d-9d05-d5a80b576d2e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 16:22:09.762041 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.762015 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:22:09.813856 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.813773 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-credential-socket\") pod \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " Jul 07 16:22:09.813856 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.813834 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-workload-certs\") pod \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " Jul 07 16:22:09.814062 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.813894 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxkpc\" (UniqueName: \"kubernetes.io/projected/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-kube-api-access-jxkpc\") pod \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " Jul 07 16:22:09.814062 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.813940 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istio-envoy\") pod \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " Jul 07 16:22:09.814062 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.813974 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istiod-ca-cert\") pod \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " Jul 07 16:22:09.814062 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.814005 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-workload-socket\") pod \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " Jul 07 16:22:09.814062 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.814035 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istio-token\") pod \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " Jul 07 16:22:09.814062 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.814046 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dda05f2e-9d64-4f4d-9d05-d5a80b576d2e" (UID: "dda05f2e-9d64-4f4d-9d05-d5a80b576d2e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:22:09.814394 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.814073 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istio-podinfo\") pod \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " Jul 07 16:22:09.814394 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.814152 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istio-data\") pod \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\" (UID: \"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e\") " Jul 07 16:22:09.814498 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.814147 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dda05f2e-9d64-4f4d-9d05-d5a80b576d2e" (UID: "dda05f2e-9d64-4f4d-9d05-d5a80b576d2e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:22:09.814498 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.814330 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dda05f2e-9d64-4f4d-9d05-d5a80b576d2e" (UID: "dda05f2e-9d64-4f4d-9d05-d5a80b576d2e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:22:09.814498 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.814338 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dda05f2e-9d64-4f4d-9d05-d5a80b576d2e" (UID: "dda05f2e-9d64-4f4d-9d05-d5a80b576d2e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:22:09.814498 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.814471 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-credential-socket\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:22:09.815348 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.815320 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istio-data" (OuterVolumeSpecName: "istio-data") pod "dda05f2e-9d64-4f4d-9d05-d5a80b576d2e" (UID: "dda05f2e-9d64-4f4d-9d05-d5a80b576d2e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:22:09.816226 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.816197 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dda05f2e-9d64-4f4d-9d05-d5a80b576d2e" (UID: "dda05f2e-9d64-4f4d-9d05-d5a80b576d2e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:22:09.816338 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.816319 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-kube-api-access-jxkpc" (OuterVolumeSpecName: "kube-api-access-jxkpc") pod "dda05f2e-9d64-4f4d-9d05-d5a80b576d2e" (UID: "dda05f2e-9d64-4f4d-9d05-d5a80b576d2e"). InnerVolumeSpecName "kube-api-access-jxkpc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:22:09.816468 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.816452 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istio-token" (OuterVolumeSpecName: "istio-token") pod "dda05f2e-9d64-4f4d-9d05-d5a80b576d2e" (UID: "dda05f2e-9d64-4f4d-9d05-d5a80b576d2e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:22:09.816576 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.816554 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dda05f2e-9d64-4f4d-9d05-d5a80b576d2e" (UID: "dda05f2e-9d64-4f4d-9d05-d5a80b576d2e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 16:22:09.857384 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.857358 2575 generic.go:356] "Generic (PLEG): container finished" podID="dda05f2e-9d64-4f4d-9d05-d5a80b576d2e" containerID="d3c41207bebccb5d8aa6d23fb0af9791beb2a7a10c18eb0c3ad0a13430e7cad8" exitCode=0 Jul 07 16:22:09.857498 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.857407 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" Jul 07 16:22:09.857498 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.857451 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" event={"ID":"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e","Type":"ContainerDied","Data":"d3c41207bebccb5d8aa6d23fb0af9791beb2a7a10c18eb0c3ad0a13430e7cad8"} Jul 07 16:22:09.857584 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.857503 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9" event={"ID":"dda05f2e-9d64-4f4d-9d05-d5a80b576d2e","Type":"ContainerDied","Data":"60f1ad499af95c20f2c4c1e01e2fe0a7cf2275c5742a98991578cbc9c87af2f5"} Jul 07 16:22:09.857584 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.857525 2575 scope.go:117] "RemoveContainer" containerID="d3c41207bebccb5d8aa6d23fb0af9791beb2a7a10c18eb0c3ad0a13430e7cad8" Jul 07 16:22:09.867074 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.867056 2575 scope.go:117] "RemoveContainer" containerID="5025ca7f513bb0a2a67cd5f18369d0eddad95d9700d992f0d1519c3a0fcca014" Jul 07 16:22:09.874845 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.874828 2575 scope.go:117] "RemoveContainer" containerID="d3c41207bebccb5d8aa6d23fb0af9791beb2a7a10c18eb0c3ad0a13430e7cad8" Jul 07 16:22:09.875141 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:22:09.875121 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3c41207bebccb5d8aa6d23fb0af9791beb2a7a10c18eb0c3ad0a13430e7cad8\": container with ID starting with d3c41207bebccb5d8aa6d23fb0af9791beb2a7a10c18eb0c3ad0a13430e7cad8 not found: ID does not exist" containerID="d3c41207bebccb5d8aa6d23fb0af9791beb2a7a10c18eb0c3ad0a13430e7cad8" Jul 07 16:22:09.875209 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.875149 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3c41207bebccb5d8aa6d23fb0af9791beb2a7a10c18eb0c3ad0a13430e7cad8"} err="failed to get container status \"d3c41207bebccb5d8aa6d23fb0af9791beb2a7a10c18eb0c3ad0a13430e7cad8\": rpc error: code = NotFound desc = could not find container \"d3c41207bebccb5d8aa6d23fb0af9791beb2a7a10c18eb0c3ad0a13430e7cad8\": container with ID starting with d3c41207bebccb5d8aa6d23fb0af9791beb2a7a10c18eb0c3ad0a13430e7cad8 not found: ID does not exist" Jul 07 16:22:09.875209 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.875168 2575 scope.go:117] "RemoveContainer" containerID="5025ca7f513bb0a2a67cd5f18369d0eddad95d9700d992f0d1519c3a0fcca014" Jul 07 16:22:09.875406 ip-10-0-131-221 kubenswrapper[2575]: E0707 16:22:09.875385 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5025ca7f513bb0a2a67cd5f18369d0eddad95d9700d992f0d1519c3a0fcca014\": container with ID starting with 5025ca7f513bb0a2a67cd5f18369d0eddad95d9700d992f0d1519c3a0fcca014 not found: ID does not exist" containerID="5025ca7f513bb0a2a67cd5f18369d0eddad95d9700d992f0d1519c3a0fcca014" Jul 07 16:22:09.875461 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.875417 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5025ca7f513bb0a2a67cd5f18369d0eddad95d9700d992f0d1519c3a0fcca014"} err="failed to get container status \"5025ca7f513bb0a2a67cd5f18369d0eddad95d9700d992f0d1519c3a0fcca014\": rpc error: code = NotFound desc = could not find container \"5025ca7f513bb0a2a67cd5f18369d0eddad95d9700d992f0d1519c3a0fcca014\": container with ID starting with 5025ca7f513bb0a2a67cd5f18369d0eddad95d9700d992f0d1519c3a0fcca014 not found: ID does not exist" Jul 07 16:22:09.883189 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.883167 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9"] Jul 07 16:22:09.888244 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.888221 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-132358-openshift-default-6c8f6d9fb7-fvlx9"] Jul 07 16:22:09.915699 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.915678 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istio-token\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:22:09.915699 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.915700 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istio-podinfo\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:22:09.915825 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.915709 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istio-data\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:22:09.915825 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.915719 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-workload-certs\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:22:09.915825 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.915727 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxkpc\" (UniqueName: \"kubernetes.io/projected/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-kube-api-access-jxkpc\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:22:09.915825 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.915737 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istio-envoy\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:22:09.915825 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.915745 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-istiod-ca-cert\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:22:09.915825 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:09.915754 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e-workload-socket\") on node \"ip-10-0-131-221.ec2.internal\" DevicePath \"\"" Jul 07 16:22:10.385878 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:10.385836 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29724000-pbjkx_1a3d17ae-cd57-4f29-9182-f21a38c4c8e4/cleanup/0.log" Jul 07 16:22:10.513518 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:10.513491 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29724015-wlp5t_ab2222c4-c95c-418a-a6a0-29900026a514/cleanup/0.log" Jul 07 16:22:11.168735 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:11.168709 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-865bb77d7b-mcj5r_1dcf30c8-4b0b-4da9-9aa4-99865a731b52/copy-manifests/0.log" Jul 07 16:22:11.178153 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:11.178131 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-865bb77d7b-mcj5r_1dcf30c8-4b0b-4da9-9aa4-99865a731b52/manager/0.log" Jul 07 16:22:11.436458 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:11.436380 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dda05f2e-9d64-4f4d-9d05-d5a80b576d2e" path="/var/lib/kubelet/pods/dda05f2e-9d64-4f4d-9d05-d5a80b576d2e/volumes" Jul 07 16:22:12.974233 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:12.974201 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-tcvfn_55f8e5a0-a309-4faa-89bc-b3748e880df2/manager/0.log" Jul 07 16:22:14.226703 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:14.226668 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-tltvx_c097cd64-a2ac-4a57-a49b-83c559abd38d/discovery/0.log" Jul 07 16:22:15.310562 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:15.310488 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-lt8x8_15d1f05e-04bf-4925-8ef5-13d3665fa6bc/main/0.log" Jul 07 16:22:15.425555 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:15.425521 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-g2dwn_d9b38285-3fcb-40f9-88c7-3541bbdc645f/main/0.log" Jul 07 16:22:23.560972 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:23.560943 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nkjmf_e207ff37-6b1c-4da4-8327-54488837f74d/global-pull-secret-syncer/0.log" Jul 07 16:22:23.836599 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:23.836527 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wtwmv_4500e026-5b93-4ce5-94ed-024a6432ab99/konnectivity-agent/0.log" Jul 07 16:22:23.903254 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:23.903225 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-221.ec2.internal_55087e15328923e7c338ebbe1c5355e7/haproxy/0.log" Jul 07 16:22:29.280191 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:29.280160 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-tcvfn_55f8e5a0-a309-4faa-89bc-b3748e880df2/manager/0.log" Jul 07 16:22:31.414480 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:31.414451 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8/alertmanager/0.log" Jul 07 16:22:31.446136 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:31.446115 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8/config-reloader/0.log" Jul 07 16:22:31.476582 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:31.476555 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8/kube-rbac-proxy-web/0.log" Jul 07 16:22:31.511974 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:31.511948 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8/kube-rbac-proxy/0.log" Jul 07 16:22:31.545576 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:31.545551 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8/kube-rbac-proxy-metric/0.log" Jul 07 16:22:31.574943 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:31.574916 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8/prom-label-proxy/0.log" Jul 07 16:22:31.608766 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:31.608746 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cae91e4f-53e4-47cd-a9f9-0bdfa6c290e8/init-config-reloader/0.log" Jul 07 16:22:31.805924 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:31.805845 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-54c5cdd484-sdptj_dfc229c2-f83c-441a-be18-cfdb40cf9097/metrics-server/0.log" Jul 07 16:22:31.835694 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:31.835672 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-xr2zq_63c46789-b6d5-4c23-823e-75fa094aaf47/monitoring-plugin/0.log" Jul 07 16:22:31.872530 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:31.872508 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6vp4d_b42f19d6-c809-45c9-bc19-0d48b7e7b355/node-exporter/0.log" Jul 07 16:22:31.900885 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:31.900836 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6vp4d_b42f19d6-c809-45c9-bc19-0d48b7e7b355/kube-rbac-proxy/0.log" Jul 07 16:22:31.929450 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:31.929425 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6vp4d_b42f19d6-c809-45c9-bc19-0d48b7e7b355/init-textfile/0.log" Jul 07 16:22:32.280294 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.280264 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eed041c2-5288-42b7-a2f7-6d8cae114196/prometheus/0.log" Jul 07 16:22:32.309480 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.309457 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eed041c2-5288-42b7-a2f7-6d8cae114196/config-reloader/0.log" Jul 07 16:22:32.345679 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.345648 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eed041c2-5288-42b7-a2f7-6d8cae114196/thanos-sidecar/0.log" Jul 07 16:22:32.373777 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.373758 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eed041c2-5288-42b7-a2f7-6d8cae114196/kube-rbac-proxy-web/0.log" Jul 07 16:22:32.408832 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.408806 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eed041c2-5288-42b7-a2f7-6d8cae114196/kube-rbac-proxy/0.log" Jul 07 16:22:32.437379 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.437355 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eed041c2-5288-42b7-a2f7-6d8cae114196/kube-rbac-proxy-thanos/0.log" Jul 07 16:22:32.469368 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.469341 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eed041c2-5288-42b7-a2f7-6d8cae114196/init-config-reloader/0.log" Jul 07 16:22:32.503306 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.503283 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-2cld9_0fc8fb5d-775b-4cf5-929d-21354c456e1b/prometheus-operator/0.log" Jul 07 16:22:32.528854 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.528833 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-2cld9_0fc8fb5d-775b-4cf5-929d-21354c456e1b/kube-rbac-proxy/0.log" Jul 07 16:22:32.591186 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.591150 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5b8596d8bf-5bw58_cc7d21d4-173e-451b-810d-56e285861599/telemeter-client/0.log" Jul 07 16:22:32.619795 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.619772 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5b8596d8bf-5bw58_cc7d21d4-173e-451b-810d-56e285861599/reload/0.log" Jul 07 16:22:32.646384 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.646350 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5b8596d8bf-5bw58_cc7d21d4-173e-451b-810d-56e285861599/kube-rbac-proxy/0.log" Jul 07 16:22:32.692086 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.692062 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-564d665bfd-zdgh9_02a7ff49-005b-4e1f-b21a-992d9828c8d8/thanos-query/0.log" Jul 07 16:22:32.725730 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.725709 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-564d665bfd-zdgh9_02a7ff49-005b-4e1f-b21a-992d9828c8d8/kube-rbac-proxy-web/0.log" Jul 07 16:22:32.777622 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.777599 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-564d665bfd-zdgh9_02a7ff49-005b-4e1f-b21a-992d9828c8d8/kube-rbac-proxy/0.log" Jul 07 16:22:32.811672 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.811648 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-564d665bfd-zdgh9_02a7ff49-005b-4e1f-b21a-992d9828c8d8/prom-label-proxy/0.log" Jul 07 16:22:32.846497 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.846431 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-564d665bfd-zdgh9_02a7ff49-005b-4e1f-b21a-992d9828c8d8/kube-rbac-proxy-rules/0.log" Jul 07 16:22:32.884745 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.884725 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-564d665bfd-zdgh9_02a7ff49-005b-4e1f-b21a-992d9828c8d8/kube-rbac-proxy-metrics/0.log" Jul 07 16:22:32.941839 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.941812 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gdvqz/perf-node-gather-daemonset-6w5dc"] Jul 07 16:22:32.942185 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.942172 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3728dfe-514c-4b21-a1f8-757825bb4a5e" containerName="istio-proxy" Jul 07 16:22:32.942230 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.942187 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3728dfe-514c-4b21-a1f8-757825bb4a5e" containerName="istio-proxy" Jul 07 16:22:32.942230 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.942197 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dda05f2e-9d64-4f4d-9d05-d5a80b576d2e" containerName="istio-proxy" Jul 07 16:22:32.942230 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.942203 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="dda05f2e-9d64-4f4d-9d05-d5a80b576d2e" containerName="istio-proxy" Jul 07 16:22:32.942230 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.942218 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="038e063a-961b-418d-9155-3b094ce011f0" containerName="maas-api" Jul 07 16:22:32.942230 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.942224 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="038e063a-961b-418d-9155-3b094ce011f0" containerName="maas-api" Jul 07 16:22:32.942386 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.942233 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3728dfe-514c-4b21-a1f8-757825bb4a5e" containerName="istio-proxy" Jul 07 16:22:32.942386 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.942238 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3728dfe-514c-4b21-a1f8-757825bb4a5e" containerName="istio-proxy" Jul 07 16:22:32.942386 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.942248 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dda05f2e-9d64-4f4d-9d05-d5a80b576d2e" containerName="istio-proxy" Jul 07 16:22:32.942386 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.942253 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="dda05f2e-9d64-4f4d-9d05-d5a80b576d2e" containerName="istio-proxy" Jul 07 16:22:32.942386 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.942307 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="038e063a-961b-418d-9155-3b094ce011f0" containerName="maas-api" Jul 07 16:22:32.942386 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.942314 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3728dfe-514c-4b21-a1f8-757825bb4a5e" containerName="istio-proxy" Jul 07 16:22:32.942386 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.942322 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="dda05f2e-9d64-4f4d-9d05-d5a80b576d2e" containerName="istio-proxy" Jul 07 16:22:32.942386 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.942327 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="dda05f2e-9d64-4f4d-9d05-d5a80b576d2e" containerName="istio-proxy" Jul 07 16:22:32.944675 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.944660 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gdvqz/perf-node-gather-daemonset-6w5dc" Jul 07 16:22:32.949520 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.949500 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gdvqz\"/\"kube-root-ca.crt\"" Jul 07 16:22:32.949617 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.949500 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gdvqz\"/\"default-dockercfg-xkzkm\"" Jul 07 16:22:32.951026 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.950790 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gdvqz\"/\"openshift-service-ca.crt\"" Jul 07 16:22:32.961700 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:32.961680 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gdvqz/perf-node-gather-daemonset-6w5dc"] Jul 07 16:22:33.029163 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:33.029126 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7d204246-26da-48ba-9b8e-b7558665f46c-podres\") pod \"perf-node-gather-daemonset-6w5dc\" (UID: \"7d204246-26da-48ba-9b8e-b7558665f46c\") " pod="openshift-must-gather-gdvqz/perf-node-gather-daemonset-6w5dc" Jul 07 16:22:33.029312 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:33.029183 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7d204246-26da-48ba-9b8e-b7558665f46c-sys\") pod \"perf-node-gather-daemonset-6w5dc\" (UID: \"7d204246-26da-48ba-9b8e-b7558665f46c\") " pod="openshift-must-gather-gdvqz/perf-node-gather-daemonset-6w5dc" Jul 07 16:22:33.029312 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:33.029207 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7d204246-26da-48ba-9b8e-b7558665f46c-proc\") pod \"perf-node-gather-daemonset-6w5dc\" (UID: \"7d204246-26da-48ba-9b8e-b7558665f46c\") " pod="openshift-must-gather-gdvqz/perf-node-gather-daemonset-6w5dc" Jul 07 16:22:33.029312 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:33.029257 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44tph\" (UniqueName: \"kubernetes.io/projected/7d204246-26da-48ba-9b8e-b7558665f46c-kube-api-access-44tph\") pod \"perf-node-gather-daemonset-6w5dc\" (UID: \"7d204246-26da-48ba-9b8e-b7558665f46c\") " pod="openshift-must-gather-gdvqz/perf-node-gather-daemonset-6w5dc" Jul 07 16:22:33.029312 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:33.029276 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7d204246-26da-48ba-9b8e-b7558665f46c-lib-modules\") pod \"perf-node-gather-daemonset-6w5dc\" (UID: \"7d204246-26da-48ba-9b8e-b7558665f46c\") " pod="openshift-must-gather-gdvqz/perf-node-gather-daemonset-6w5dc" Jul 07 16:22:33.130303 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:33.130222 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7d204246-26da-48ba-9b8e-b7558665f46c-podres\") pod \"perf-node-gather-daemonset-6w5dc\" (UID: \"7d204246-26da-48ba-9b8e-b7558665f46c\") " pod="openshift-must-gather-gdvqz/perf-node-gather-daemonset-6w5dc" Jul 07 16:22:33.130303 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:33.130267 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7d204246-26da-48ba-9b8e-b7558665f46c-sys\") pod \"perf-node-gather-daemonset-6w5dc\" (UID: \"7d204246-26da-48ba-9b8e-b7558665f46c\") " pod="openshift-must-gather-gdvqz/perf-node-gather-daemonset-6w5dc" Jul 07 16:22:33.130303 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:33.130287 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7d204246-26da-48ba-9b8e-b7558665f46c-proc\") pod \"perf-node-gather-daemonset-6w5dc\" (UID: \"7d204246-26da-48ba-9b8e-b7558665f46c\") " pod="openshift-must-gather-gdvqz/perf-node-gather-daemonset-6w5dc" Jul 07 16:22:33.130505 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:33.130356 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44tph\" (UniqueName: \"kubernetes.io/projected/7d204246-26da-48ba-9b8e-b7558665f46c-kube-api-access-44tph\") pod \"perf-node-gather-daemonset-6w5dc\" (UID: \"7d204246-26da-48ba-9b8e-b7558665f46c\") " pod="openshift-must-gather-gdvqz/perf-node-gather-daemonset-6w5dc" Jul 07 16:22:33.130505 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:33.130365 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7d204246-26da-48ba-9b8e-b7558665f46c-sys\") pod \"perf-node-gather-daemonset-6w5dc\" (UID: \"7d204246-26da-48ba-9b8e-b7558665f46c\") " pod="openshift-must-gather-gdvqz/perf-node-gather-daemonset-6w5dc" Jul 07 16:22:33.130505 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:33.130395 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7d204246-26da-48ba-9b8e-b7558665f46c-podres\") pod \"perf-node-gather-daemonset-6w5dc\" (UID: \"7d204246-26da-48ba-9b8e-b7558665f46c\") " pod="openshift-must-gather-gdvqz/perf-node-gather-daemonset-6w5dc" Jul 07 16:22:33.130505 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:33.130414 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7d204246-26da-48ba-9b8e-b7558665f46c-lib-modules\") pod \"perf-node-gather-daemonset-6w5dc\" (UID: \"7d204246-26da-48ba-9b8e-b7558665f46c\") " pod="openshift-must-gather-gdvqz/perf-node-gather-daemonset-6w5dc" Jul 07 16:22:33.130505 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:33.130425 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7d204246-26da-48ba-9b8e-b7558665f46c-proc\") pod \"perf-node-gather-daemonset-6w5dc\" (UID: \"7d204246-26da-48ba-9b8e-b7558665f46c\") " pod="openshift-must-gather-gdvqz/perf-node-gather-daemonset-6w5dc" Jul 07 16:22:33.130505 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:33.130500 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7d204246-26da-48ba-9b8e-b7558665f46c-lib-modules\") pod \"perf-node-gather-daemonset-6w5dc\" (UID: \"7d204246-26da-48ba-9b8e-b7558665f46c\") " pod="openshift-must-gather-gdvqz/perf-node-gather-daemonset-6w5dc" Jul 07 16:22:33.144510 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:33.144483 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44tph\" (UniqueName: \"kubernetes.io/projected/7d204246-26da-48ba-9b8e-b7558665f46c-kube-api-access-44tph\") pod \"perf-node-gather-daemonset-6w5dc\" (UID: \"7d204246-26da-48ba-9b8e-b7558665f46c\") " pod="openshift-must-gather-gdvqz/perf-node-gather-daemonset-6w5dc" Jul 07 16:22:33.254847 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:33.254797 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gdvqz/perf-node-gather-daemonset-6w5dc" Jul 07 16:22:33.389065 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:33.388986 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gdvqz/perf-node-gather-daemonset-6w5dc"] Jul 07 16:22:33.392231 ip-10-0-131-221 kubenswrapper[2575]: W0707 16:22:33.392202 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7d204246_26da_48ba_9b8e_b7558665f46c.slice/crio-49cfa39524d8be196a527057e20a441717cebe3c6e87fb49086e1d00e7ad48cc WatchSource:0}: Error finding container 49cfa39524d8be196a527057e20a441717cebe3c6e87fb49086e1d00e7ad48cc: Status 404 returned error can't find the container with id 49cfa39524d8be196a527057e20a441717cebe3c6e87fb49086e1d00e7ad48cc Jul 07 16:22:33.974666 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:33.974633 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gdvqz/perf-node-gather-daemonset-6w5dc" event={"ID":"7d204246-26da-48ba-9b8e-b7558665f46c","Type":"ContainerStarted","Data":"fdaa3703de87e24ba708770cfaf10370c920b1a6dc8fb3aa4791f6fa05825424"} Jul 07 16:22:33.975073 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:33.974683 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gdvqz/perf-node-gather-daemonset-6w5dc" event={"ID":"7d204246-26da-48ba-9b8e-b7558665f46c","Type":"ContainerStarted","Data":"49cfa39524d8be196a527057e20a441717cebe3c6e87fb49086e1d00e7ad48cc"} Jul 07 16:22:33.975073 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:33.974748 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gdvqz/perf-node-gather-daemonset-6w5dc" Jul 07 16:22:33.996503 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:33.996454 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gdvqz/perf-node-gather-daemonset-6w5dc" podStartSLOduration=1.996440304 podStartE2EDuration="1.996440304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 16:22:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:22:33.995017104 +0000 UTC m=+2441.090298184" watchObservedRunningTime="2026-07-07 16:22:33.996440304 +0000 UTC m=+2441.091721382" Jul 07 16:22:34.729220 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:34.729193 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nfh8b_162f95df-a98f-49a6-a91e-689a8e22be46/console-operator/1.log" Jul 07 16:22:34.733380 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:34.733359 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nfh8b_162f95df-a98f-49a6-a91e-689a8e22be46/console-operator/2.log" Jul 07 16:22:37.123800 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:37.123771 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m2fcv_73fd3204-d75e-41e7-ae19-734ddf9d80ae/dns/0.log" Jul 07 16:22:37.158906 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:37.158877 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m2fcv_73fd3204-d75e-41e7-ae19-734ddf9d80ae/kube-rbac-proxy/0.log" Jul 07 16:22:37.188149 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:37.188123 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mpvp5_56652ce6-f2fe-44ee-a263-72343b20fafe/dns-node-resolver/0.log" Jul 07 16:22:37.919226 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:37.919199 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9ng9d_0efee2cb-622f-4f59-a8a4-1eecfc38ead4/node-ca/0.log" Jul 07 16:22:39.099377 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:39.099344 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-tltvx_c097cd64-a2ac-4a57-a49b-83c559abd38d/discovery/0.log" Jul 07 16:22:39.994963 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:39.994923 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gdvqz/perf-node-gather-daemonset-6w5dc" Jul 07 16:22:40.204123 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:40.204098 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xb8r5_19092adf-0ed9-4061-9a0a-bbc7dc75bc40/serve-healthcheck-canary/0.log" Jul 07 16:22:41.041209 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:41.041180 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tv4kj_2b5661bd-f1c4-4f6e-89a8-6e84a49fb953/kube-rbac-proxy/0.log" Jul 07 16:22:41.072671 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:41.072651 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tv4kj_2b5661bd-f1c4-4f6e-89a8-6e84a49fb953/exporter/0.log" Jul 07 16:22:41.100669 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:41.100646 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tv4kj_2b5661bd-f1c4-4f6e-89a8-6e84a49fb953/extractor/0.log" Jul 07 16:22:43.576377 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:43.576351 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-tltvx_c097cd64-a2ac-4a57-a49b-83c559abd38d/discovery/0.log" Jul 07 16:22:43.748086 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:43.748009 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29724000-pbjkx_1a3d17ae-cd57-4f29-9182-f21a38c4c8e4/cleanup/0.log" Jul 07 16:22:43.786542 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:43.786510 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29724015-wlp5t_ab2222c4-c95c-418a-a6a0-29900026a514/cleanup/0.log" Jul 07 16:22:44.162522 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:44.162498 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-865bb77d7b-mcj5r_1dcf30c8-4b0b-4da9-9aa4-99865a731b52/manager/0.log" Jul 07 16:22:44.201281 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:44.201252 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-865bb77d7b-mcj5r_1dcf30c8-4b0b-4da9-9aa4-99865a731b52/copy-manifests/0.log" Jul 07 16:22:45.104322 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:45.104290 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-tltvx_c097cd64-a2ac-4a57-a49b-83c559abd38d/discovery/0.log" Jul 07 16:22:45.681897 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:45.681810 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-86b7f856ff-jbr6g_1e471450-c3bf-4332-9fda-fd14594f7150/manager/0.log" Jul 07 16:22:50.725526 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:50.725493 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-dx8vc_da62451e-981a-4473-85a5-ec61606dc856/migrator/0.log" Jul 07 16:22:50.750762 ip-10-0-131-221 kubenswrapper[2575]: I0707 16:22:50.750730 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-dx8vc_da62451e-981a-4473-85a5-ec61606dc856/graceful-termination/0.log"