Jul 05 13:18:33.545095 ip-10-0-141-119 systemd[1]: Starting Kubernetes Kubelet... Jul 05 13:18:33.976372 ip-10-0-141-119 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 05 13:18:33.976372 ip-10-0-141-119 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 05 13:18:33.976372 ip-10-0-141-119 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 05 13:18:33.976372 ip-10-0-141-119 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 05 13:18:33.976372 ip-10-0-141-119 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 05 13:18:33.977393 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.977274 2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 05 13:18:33.983117 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983095 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:18:33.983117 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983117 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:18:33.983192 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983121 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:18:33.983192 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983126 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:18:33.983192 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983129 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:18:33.983192 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983132 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:18:33.983192 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983135 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:18:33.983192 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983137 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:18:33.983192 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983140 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:18:33.983192 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983143 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:18:33.983192 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983146 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:18:33.983192 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983149 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:18:33.983192 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983151 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:18:33.983192 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983154 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:18:33.983192 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983157 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:18:33.983192 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983159 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:18:33.983192 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983162 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:18:33.983192 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983171 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:18:33.983192 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983173 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:18:33.983192 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983176 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:18:33.983192 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983179 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:18:33.983646 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983181 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:18:33.983646 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983183 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:18:33.983646 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983186 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:18:33.983646 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983188 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:18:33.983646 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983191 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:18:33.983646 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983194 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:18:33.983646 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983196 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:18:33.983646 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983200 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:18:33.983646 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983202 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:18:33.983646 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983205 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:18:33.983646 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983207 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:18:33.983646 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983210 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:18:33.983646 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983213 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:18:33.983646 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983215 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:18:33.983646 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983220 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:18:33.983646 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983224 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:18:33.983646 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983226 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:18:33.983646 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983229 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:18:33.983646 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983231 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:18:33.983646 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983234 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:18:33.984171 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983236 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:18:33.984171 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983238 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:18:33.984171 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983241 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:18:33.984171 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983243 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:18:33.984171 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983254 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:18:33.984171 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983257 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:18:33.984171 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983259 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:18:33.984171 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983261 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:18:33.984171 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983264 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:18:33.984171 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983267 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:18:33.984171 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983269 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:18:33.984171 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983272 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:18:33.984171 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983274 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:18:33.984171 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983279 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:18:33.984171 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983281 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:18:33.984171 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983284 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:18:33.984171 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983287 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:18:33.984171 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983289 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:18:33.984171 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983291 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:18:33.984171 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983294 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:18:33.984654 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983298 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:18:33.984654 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983303 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:18:33.984654 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983306 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:18:33.984654 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983309 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:18:33.984654 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983312 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:18:33.984654 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983315 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:18:33.984654 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983317 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:18:33.984654 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983320 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:18:33.984654 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983323 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:18:33.984654 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983325 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:18:33.984654 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983329 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:18:33.984654 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983331 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:18:33.984654 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983334 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:18:33.984654 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983336 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:18:33.984654 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983338 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:18:33.984654 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983341 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:18:33.984654 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983343 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:18:33.984654 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983346 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:18:33.984654 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983349 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:18:33.985133 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983352 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:18:33.985133 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983354 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:18:33.985133 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983357 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:18:33.985133 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983360 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:18:33.985133 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983362 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:18:33.985133 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983365 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:18:33.985133 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983368 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:18:33.985133 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983370 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:18:33.985133 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983373 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:18:33.985133 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983376 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:18:33.985133 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983378 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:18:33.985133 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983380 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:18:33.985133 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983383 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:18:33.985133 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983385 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:18:33.985133 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983388 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:18:33.985133 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983390 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:18:33.985133 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983394 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:18:33.985133 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983397 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:18:33.985133 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.983400 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:18:33.985133 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984526 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:18:33.985933 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984534 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:18:33.985933 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984537 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:18:33.985933 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984540 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:18:33.985933 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984543 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:18:33.985933 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984546 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:18:33.985933 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984548 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:18:33.985933 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984551 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:18:33.985933 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984553 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:18:33.985933 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984556 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:18:33.985933 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984558 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:18:33.985933 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984561 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:18:33.985933 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984565 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:18:33.985933 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984569 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:18:33.985933 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984572 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:18:33.985933 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984575 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:18:33.985933 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984577 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:18:33.985933 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984581 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:18:33.985933 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984583 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:18:33.985933 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984587 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:18:33.985933 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984589 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:18:33.986775 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984591 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:18:33.986775 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984594 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:18:33.986775 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984596 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:18:33.986775 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984599 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:18:33.986775 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984602 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:18:33.986775 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984604 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:18:33.986775 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984608 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:18:33.986775 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984611 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:18:33.986775 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984614 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:18:33.986775 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984616 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:18:33.986775 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984619 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:18:33.986775 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984621 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:18:33.986775 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984624 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:18:33.986775 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984627 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:18:33.986775 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984629 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:18:33.986775 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984632 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:18:33.986775 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984634 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:18:33.986775 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984637 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:18:33.986775 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984639 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:18:33.987650 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984642 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:18:33.987650 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984644 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:18:33.987650 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984647 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:18:33.987650 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984649 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:18:33.987650 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984652 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:18:33.987650 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984654 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:18:33.987650 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984657 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:18:33.987650 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984659 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:18:33.987650 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984662 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:18:33.987650 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984664 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:18:33.987650 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984668 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:18:33.987650 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984671 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:18:33.987650 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984674 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:18:33.987650 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984676 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:18:33.987650 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984679 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:18:33.987650 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984681 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:18:33.987650 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984684 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:18:33.987650 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984687 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:18:33.987650 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984689 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:18:33.987650 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984691 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:18:33.988519 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984694 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:18:33.988519 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984696 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:18:33.988519 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984699 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:18:33.988519 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984701 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:18:33.988519 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984703 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:18:33.988519 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984706 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:18:33.988519 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984709 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:18:33.988519 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984711 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:18:33.988519 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984714 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:18:33.988519 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984716 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:18:33.988519 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984718 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:18:33.988519 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984721 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:18:33.988519 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984723 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:18:33.988519 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984726 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:18:33.988519 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984728 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:18:33.988519 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984731 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:18:33.988519 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984733 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:18:33.988519 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984755 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:18:33.988519 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984760 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:18:33.989137 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984762 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:18:33.989137 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984765 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:18:33.989137 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984767 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:18:33.989137 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984770 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:18:33.989137 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984776 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:18:33.989137 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984778 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:18:33.989137 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984781 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:18:33.989137 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984783 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:18:33.989137 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984786 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:18:33.989137 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984790 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:18:33.989137 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984793 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:18:33.989137 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984796 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:18:33.989137 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984799 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:18:33.989137 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984802 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:18:33.989137 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984805 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:18:33.989137 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984808 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:18:33.989137 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984810 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:18:33.989137 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984813 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:18:33.989137 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984816 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:18:33.989644 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.984818 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:18:33.989644 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986716 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:18:33.989644 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986760 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:18:33.989644 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986768 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:18:33.989644 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986773 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:18:33.989644 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986777 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:18:33.989644 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986782 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:18:33.989644 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986786 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:18:33.989644 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986791 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:18:33.989644 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986795 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:18:33.989644 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986799 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:18:33.989644 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986804 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:18:33.989644 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986808 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:18:33.989644 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986813 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:18:33.989644 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986817 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:18:33.989644 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986821 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:18:33.989644 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986826 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:18:33.989644 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986831 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:18:33.989644 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986835 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:18:33.990212 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986840 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:18:33.990212 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986844 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:18:33.990212 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986848 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:18:33.990212 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986853 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:18:33.990212 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986856 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:18:33.990212 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986861 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:18:33.990212 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986866 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:18:33.990212 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986870 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:18:33.990212 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986876 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:18:33.990212 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986880 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:18:33.990212 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986884 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:18:33.990212 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986889 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:18:33.990212 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986893 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:18:33.990212 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986896 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:18:33.990212 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986901 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:18:33.990212 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986907 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:18:33.990212 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986912 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:18:33.990212 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986916 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:18:33.990212 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986920 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:18:33.990805 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986924 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:18:33.990805 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986929 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:18:33.990805 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986933 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:18:33.990805 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986938 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:18:33.990805 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986942 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:18:33.990805 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986946 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:18:33.990805 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986951 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:18:33.990805 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986956 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:18:33.990805 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986960 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:18:33.990805 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986964 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:18:33.990805 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986968 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:18:33.990805 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986973 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:18:33.990805 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986977 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:18:33.990805 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986983 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:18:33.990805 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986988 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:18:33.990805 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986992 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:18:33.990805 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.986996 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:18:33.990805 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987001 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:18:33.990805 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987008 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:18:33.991430 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987015 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:18:33.991430 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987020 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:18:33.991430 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987024 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:18:33.991430 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987029 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:18:33.991430 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987033 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:18:33.991430 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987037 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:18:33.991430 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987042 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:18:33.991430 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987046 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:18:33.991430 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987050 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:18:33.991430 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987055 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:18:33.991430 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987059 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:18:33.991430 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987066 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:18:33.991430 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987072 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:18:33.991430 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987078 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:18:33.991430 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987082 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:18:33.991430 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987087 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:18:33.991430 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987092 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:18:33.991430 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987096 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:18:33.991430 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987101 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:18:33.991923 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987105 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:18:33.991923 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987110 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:18:33.991923 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987115 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:18:33.991923 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987120 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:18:33.991923 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987127 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:18:33.991923 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987132 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:18:33.991923 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987136 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:18:33.991923 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987140 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:18:33.991923 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987145 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:18:33.991923 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987149 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:18:33.991923 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987153 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:18:33.991923 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987158 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:18:33.991923 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987162 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:18:33.991923 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987166 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:18:33.991923 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987171 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:18:33.991923 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987175 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:18:33.991923 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987179 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:18:33.991923 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987184 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:18:33.991923 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987188 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:18:33.991923 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987192 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:18:33.992400 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987196 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:18:33.992400 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987201 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:18:33.992400 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987205 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:18:33.992400 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.987210 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:18:33.992400 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987319 2567 flags.go:64] FLAG: --address="0.0.0.0" Jul 05 13:18:33.992400 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987332 2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 05 13:18:33.992400 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987342 2567 flags.go:64] FLAG: --anonymous-auth="true" Jul 05 13:18:33.992400 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987349 2567 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 05 13:18:33.992400 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987361 2567 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 05 13:18:33.992400 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987366 2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 05 13:18:33.992400 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987373 2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 05 13:18:33.992400 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987380 2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 05 13:18:33.992400 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987385 2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 05 13:18:33.992400 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987389 2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 05 13:18:33.992400 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987395 2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 05 13:18:33.992400 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987400 2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 05 13:18:33.992400 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987405 2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 05 13:18:33.992400 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987410 2567 flags.go:64] FLAG: --cgroup-root="" Jul 05 13:18:33.992400 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987415 2567 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 05 13:18:33.992400 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987419 2567 flags.go:64] FLAG: --client-ca-file="" Jul 05 13:18:33.992400 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987424 2567 flags.go:64] FLAG: --cloud-provider="external" Jul 05 13:18:33.992400 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987430 2567 flags.go:64] FLAG: --cluster-dns="[]" Jul 05 13:18:33.992400 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987437 2567 flags.go:64] FLAG: --cluster-domain="" Jul 05 13:18:33.993044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987442 2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 05 13:18:33.993044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987447 2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 05 13:18:33.993044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987452 2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 05 13:18:33.993044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987458 2567 flags.go:64] FLAG: --container-log-max-files="5" Jul 05 13:18:33.993044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987464 2567 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 05 13:18:33.993044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987468 2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 05 13:18:33.993044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987473 2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 05 13:18:33.993044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987478 2567 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 05 13:18:33.993044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987483 2567 flags.go:64] FLAG: --contention-profiling="false" Jul 05 13:18:33.993044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987487 2567 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 05 13:18:33.993044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987492 2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 05 13:18:33.993044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987498 2567 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 05 13:18:33.993044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987502 2567 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 05 13:18:33.993044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987509 2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 05 13:18:33.993044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987513 2567 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 05 13:18:33.993044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987518 2567 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 05 13:18:33.993044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987522 2567 flags.go:64] FLAG: --enable-load-reader="false" Jul 05 13:18:33.993044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987527 2567 flags.go:64] FLAG: --enable-server="true" Jul 05 13:18:33.993044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987532 2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 05 13:18:33.993044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987539 2567 flags.go:64] FLAG: --event-burst="100" Jul 05 13:18:33.993044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987545 2567 flags.go:64] FLAG: --event-qps="50" Jul 05 13:18:33.993044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987549 2567 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 05 13:18:33.993044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987554 2567 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 05 13:18:33.993044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987558 2567 flags.go:64] FLAG: --eviction-hard="" Jul 05 13:18:33.993635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987564 2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 05 13:18:33.993635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987569 2567 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 05 13:18:33.993635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987574 2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 05 13:18:33.993635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987579 2567 flags.go:64] FLAG: --eviction-soft="" Jul 05 13:18:33.993635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987584 2567 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 05 13:18:33.993635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987589 2567 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 05 13:18:33.993635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987593 2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 05 13:18:33.993635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987599 2567 flags.go:64] FLAG: --experimental-mounter-path="" Jul 05 13:18:33.993635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987603 2567 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 05 13:18:33.993635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987608 2567 flags.go:64] FLAG: --fail-swap-on="true" Jul 05 13:18:33.993635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987612 2567 flags.go:64] FLAG: --feature-gates="" Jul 05 13:18:33.993635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987619 2567 flags.go:64] FLAG: --file-check-frequency="20s" Jul 05 13:18:33.993635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987625 2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 05 13:18:33.993635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987629 2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 05 13:18:33.993635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987634 2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 05 13:18:33.993635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987639 2567 flags.go:64] FLAG: --healthz-port="10248" Jul 05 13:18:33.993635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987644 2567 flags.go:64] FLAG: --help="false" Jul 05 13:18:33.993635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987648 2567 flags.go:64] FLAG: --hostname-override="ip-10-0-141-119.ec2.internal" Jul 05 13:18:33.993635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987653 2567 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 05 13:18:33.993635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987657 2567 flags.go:64] FLAG: --http-check-frequency="20s" Jul 05 13:18:33.993635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987661 2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 05 13:18:33.993635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987666 2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 05 13:18:33.993635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987672 2567 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 05 13:18:33.994427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987676 2567 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 05 13:18:33.994427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987681 2567 flags.go:64] FLAG: --image-service-endpoint="" Jul 05 13:18:33.994427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987685 2567 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 05 13:18:33.994427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987690 2567 flags.go:64] FLAG: --kube-api-burst="100" Jul 05 13:18:33.994427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987695 2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 05 13:18:33.994427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987700 2567 flags.go:64] FLAG: --kube-api-qps="50" Jul 05 13:18:33.994427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987704 2567 flags.go:64] FLAG: --kube-reserved="" Jul 05 13:18:33.994427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987709 2567 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 05 13:18:33.994427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987713 2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 05 13:18:33.994427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987718 2567 flags.go:64] FLAG: --kubelet-cgroups="" Jul 05 13:18:33.994427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987723 2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 05 13:18:33.994427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987727 2567 flags.go:64] FLAG: --lock-file="" Jul 05 13:18:33.994427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987731 2567 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 05 13:18:33.994427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987760 2567 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 05 13:18:33.994427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987766 2567 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 05 13:18:33.994427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987775 2567 flags.go:64] FLAG: --log-json-split-stream="false" Jul 05 13:18:33.994427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987780 2567 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 05 13:18:33.994427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987785 2567 flags.go:64] FLAG: --log-text-split-stream="false" Jul 05 13:18:33.994427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987789 2567 flags.go:64] FLAG: --logging-format="text" Jul 05 13:18:33.994427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987794 2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 05 13:18:33.994427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987800 2567 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 05 13:18:33.994427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987805 2567 flags.go:64] FLAG: --manifest-url="" Jul 05 13:18:33.994427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987809 2567 flags.go:64] FLAG: --manifest-url-header="" Jul 05 13:18:33.994427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987817 2567 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 05 13:18:33.994427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987822 2567 flags.go:64] FLAG: --max-open-files="1000000" Jul 05 13:18:33.995066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987828 2567 flags.go:64] FLAG: --max-pods="110" Jul 05 13:18:33.995066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987832 2567 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 05 13:18:33.995066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987836 2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 05 13:18:33.995066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987845 2567 flags.go:64] FLAG: --memory-manager-policy="None" Jul 05 13:18:33.995066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987849 2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 05 13:18:33.995066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987854 2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 05 13:18:33.995066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987859 2567 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 05 13:18:33.995066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987864 2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 05 13:18:33.995066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987876 2567 flags.go:64] FLAG: --node-status-max-images="50" Jul 05 13:18:33.995066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987881 2567 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 05 13:18:33.995066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987886 2567 flags.go:64] FLAG: --oom-score-adj="-999" Jul 05 13:18:33.995066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987891 2567 flags.go:64] FLAG: --pod-cidr="" Jul 05 13:18:33.995066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987895 2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 05 13:18:33.995066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987905 2567 flags.go:64] FLAG: --pod-manifest-path="" Jul 05 13:18:33.995066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987911 2567 flags.go:64] FLAG: --pod-max-pids="-1" Jul 05 13:18:33.995066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987916 2567 flags.go:64] FLAG: --pods-per-core="0" Jul 05 13:18:33.995066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987921 2567 flags.go:64] FLAG: --port="10250" Jul 05 13:18:33.995066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987926 2567 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 05 13:18:33.995066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987931 2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ca057667b53f5340" Jul 05 13:18:33.995066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987936 2567 flags.go:64] FLAG: --qos-reserved="" Jul 05 13:18:33.995066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987941 2567 flags.go:64] FLAG: --read-only-port="10255" Jul 05 13:18:33.995066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987946 2567 flags.go:64] FLAG: --register-node="true" Jul 05 13:18:33.995066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987950 2567 flags.go:64] FLAG: --register-with-taints="" Jul 05 13:18:33.995066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987956 2567 flags.go:64] FLAG: --registry-burst="10" Jul 05 13:18:33.995658 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987961 2567 flags.go:64] FLAG: --registry-qps="5" Jul 05 13:18:33.995658 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987966 2567 flags.go:64] FLAG: --reserved-cpus="" Jul 05 13:18:33.995658 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987972 2567 flags.go:64] FLAG: --reserved-memory="" Jul 05 13:18:33.995658 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987978 2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 05 13:18:33.995658 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987983 2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 05 13:18:33.995658 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987988 2567 flags.go:64] FLAG: --rotate-certificates="false" Jul 05 13:18:33.995658 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987992 2567 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 05 13:18:33.995658 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.987997 2567 flags.go:64] FLAG: --runonce="false" Jul 05 13:18:33.995658 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988002 2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 05 13:18:33.995658 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988006 2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 05 13:18:33.995658 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988011 2567 flags.go:64] FLAG: --seccomp-default="false" Jul 05 13:18:33.995658 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988016 2567 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 05 13:18:33.995658 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988022 2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 05 13:18:33.995658 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988027 2567 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 05 13:18:33.995658 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988031 2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 05 13:18:33.995658 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988036 2567 flags.go:64] FLAG: --storage-driver-password="root" Jul 05 13:18:33.995658 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988041 2567 flags.go:64] FLAG: --storage-driver-secure="false" Jul 05 13:18:33.995658 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988045 2567 flags.go:64] FLAG: --storage-driver-table="stats" Jul 05 13:18:33.995658 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988050 2567 flags.go:64] FLAG: --storage-driver-user="root" Jul 05 13:18:33.995658 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988055 2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 05 13:18:33.995658 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988060 2567 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 05 13:18:33.995658 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988064 2567 flags.go:64] FLAG: --system-cgroups="" Jul 05 13:18:33.995658 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988069 2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 05 13:18:33.995658 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988077 2567 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 05 13:18:33.995658 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988082 2567 flags.go:64] FLAG: --tls-cert-file="" Jul 05 13:18:33.996372 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988086 2567 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 05 13:18:33.996372 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988093 2567 flags.go:64] FLAG: --tls-min-version="" Jul 05 13:18:33.996372 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988098 2567 flags.go:64] FLAG: --tls-private-key-file="" Jul 05 13:18:33.996372 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988102 2567 flags.go:64] FLAG: --topology-manager-policy="none" Jul 05 13:18:33.996372 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988107 2567 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 05 13:18:33.996372 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988112 2567 flags.go:64] FLAG: --topology-manager-scope="container" Jul 05 13:18:33.996372 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988116 2567 flags.go:64] FLAG: --v="2" Jul 05 13:18:33.996372 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988123 2567 flags.go:64] FLAG: --version="false" Jul 05 13:18:33.996372 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988130 2567 flags.go:64] FLAG: --vmodule="" Jul 05 13:18:33.996372 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988137 2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 05 13:18:33.996372 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.988144 2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 05 13:18:33.996372 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988339 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:18:33.996372 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988346 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:18:33.996372 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988351 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:18:33.996372 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988356 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:18:33.996372 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988360 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:18:33.996372 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988365 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:18:33.996372 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988369 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:18:33.996372 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988374 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:18:33.996372 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988380 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:18:33.996372 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988384 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:18:33.996372 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988388 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:18:33.996947 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988392 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:18:33.996947 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988397 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:18:33.996947 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988401 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:18:33.996947 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988405 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:18:33.996947 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988409 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:18:33.996947 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988413 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:18:33.996947 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988417 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:18:33.996947 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988422 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:18:33.996947 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988426 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:18:33.996947 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988430 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:18:33.996947 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988434 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:18:33.996947 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988438 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:18:33.996947 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988443 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:18:33.996947 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988447 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:18:33.996947 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988451 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:18:33.996947 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988455 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:18:33.996947 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988459 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:18:33.996947 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988463 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:18:33.996947 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988467 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:18:33.996947 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988471 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:18:33.997427 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988479 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:18:33.997427 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988483 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:18:33.997427 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988487 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:18:33.997427 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988491 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:18:33.997427 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988495 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:18:33.997427 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988500 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:18:33.997427 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988504 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:18:33.997427 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988509 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:18:33.997427 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988513 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:18:33.997427 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988519 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:18:33.997427 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988523 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:18:33.997427 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988529 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:18:33.997427 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988535 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:18:33.997427 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988539 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:18:33.997427 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988543 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:18:33.997427 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988550 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:18:33.997427 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988556 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:18:33.997427 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988561 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:18:33.997427 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988566 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:18:33.997963 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988570 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:18:33.997963 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988575 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:18:33.997963 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988579 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:18:33.997963 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988584 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:18:33.997963 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988588 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:18:33.997963 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988592 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:18:33.997963 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988596 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:18:33.997963 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988600 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:18:33.997963 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988605 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:18:33.997963 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988609 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:18:33.997963 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988613 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:18:33.997963 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988618 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:18:33.997963 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988622 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:18:33.997963 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988628 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:18:33.997963 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988632 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:18:33.997963 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988637 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:18:33.997963 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988641 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:18:33.997963 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988645 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:18:33.997963 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988649 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:18:33.997963 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988654 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:18:33.998463 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988660 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:18:33.998463 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988665 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:18:33.998463 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988671 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:18:33.998463 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988675 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:18:33.998463 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988680 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:18:33.998463 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988685 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:18:33.998463 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988690 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:18:33.998463 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988694 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:18:33.998463 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988699 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:18:33.998463 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988704 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:18:33.998463 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988708 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:18:33.998463 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988712 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:18:33.998463 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988716 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:18:33.998463 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988720 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:18:33.998463 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988725 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:18:33.998463 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988729 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:18:33.998463 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988734 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:18:33.998463 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988756 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:18:33.998463 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988760 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:18:33.998945 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988764 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:18:33.998945 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988768 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:18:33.998945 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988773 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:18:33.998945 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988777 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:18:33.998945 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988781 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:18:33.998945 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988785 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:18:33.998945 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988790 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:18:33.998945 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988795 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:18:33.998945 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988800 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:18:33.998945 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.988804 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:18:33.998945 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.989582 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 05 13:18:33.998945 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.997636 2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 05 13:18:33.998945 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.997655 2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 05 13:18:33.998945 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.997676 2567 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 05 13:18:33.998945 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.997683 2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 05 13:18:33.999313 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997762 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:18:33.999313 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997768 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:18:33.999313 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997772 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:18:33.999313 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997775 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:18:33.999313 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997778 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:18:33.999313 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997781 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:18:33.999313 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997784 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:18:33.999313 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997786 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:18:33.999313 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997790 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:18:33.999313 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997792 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:18:33.999313 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997795 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:18:33.999313 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997798 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:18:33.999313 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997801 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:18:33.999313 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997805 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:18:33.999313 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997809 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:18:33.999313 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997812 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:18:33.999313 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997815 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:18:33.999313 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997817 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:18:33.999313 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997828 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:18:33.999864 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997830 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:18:33.999864 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997833 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:18:33.999864 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997836 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:18:33.999864 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997838 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:18:33.999864 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997841 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:18:33.999864 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997843 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:18:33.999864 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997846 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:18:33.999864 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997849 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:18:33.999864 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997851 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:18:33.999864 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997854 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:18:33.999864 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997858 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:18:33.999864 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997861 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:18:33.999864 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997864 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:18:33.999864 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997866 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:18:33.999864 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997869 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:18:33.999864 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997871 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:18:33.999864 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997874 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:18:33.999864 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997876 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:18:33.999864 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997879 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:18:33.999864 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997881 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:18:34.000356 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997884 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:18:34.000356 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997886 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:18:34.000356 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997889 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:18:34.000356 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997892 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:18:34.000356 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997894 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:18:34.000356 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997896 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:18:34.000356 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997899 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:18:34.000356 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997901 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:18:34.000356 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997904 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:18:34.000356 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997906 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:18:34.000356 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997908 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:18:34.000356 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997911 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:18:34.000356 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997914 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:18:34.000356 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997916 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:18:34.000356 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997919 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:18:34.000356 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997921 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:18:34.000356 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997924 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:18:34.000356 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997926 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:18:34.000356 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997928 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:18:34.000356 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997931 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:18:34.000868 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997933 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:18:34.000868 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997936 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:18:34.000868 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997939 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:18:34.000868 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997943 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:18:34.000868 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997946 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:18:34.000868 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997948 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:18:34.000868 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997951 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:18:34.000868 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997953 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:18:34.000868 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997956 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:18:34.000868 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997958 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:18:34.000868 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997961 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:18:34.000868 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997963 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:18:34.000868 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997966 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:18:34.000868 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997968 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:18:34.000868 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997972 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:18:34.000868 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997974 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:18:34.000868 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997977 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:18:34.000868 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997980 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:18:34.000868 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997982 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:18:34.000868 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997985 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:18:34.001386 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997988 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:18:34.001386 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997990 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:18:34.001386 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997994 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:18:34.001386 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.997998 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:18:34.001386 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998002 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:18:34.001386 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998004 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:18:34.001386 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998007 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:18:34.001386 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998009 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:18:34.001386 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998012 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:18:34.001386 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998014 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:18:34.001386 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998017 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:18:34.001386 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998019 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:18:34.001386 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998022 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:18:34.001386 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998025 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:18:34.001386 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998028 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:18:34.001386 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998031 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:18:34.001386 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998034 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:18:34.001386 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998036 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:18:34.001386 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998040 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:18:34.001881 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998043 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:18:34.001881 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.998048 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 05 13:18:34.001881 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998170 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:18:34.001881 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998176 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:18:34.001881 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998179 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:18:34.001881 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998182 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:18:34.001881 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998185 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:18:34.001881 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998188 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:18:34.001881 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998190 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:18:34.001881 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998193 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:18:34.001881 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998195 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:18:34.001881 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998197 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:18:34.001881 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998200 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:18:34.001881 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998202 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:18:34.001881 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998205 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:18:34.002244 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998208 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:18:34.002244 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998210 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:18:34.002244 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998213 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:18:34.002244 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998216 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:18:34.002244 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998219 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:18:34.002244 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998221 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:18:34.002244 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998224 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:18:34.002244 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998226 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:18:34.002244 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998229 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:18:34.002244 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998231 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:18:34.002244 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998233 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:18:34.002244 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998236 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:18:34.002244 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998239 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:18:34.002244 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998242 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:18:34.002244 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998247 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:18:34.002244 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998250 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:18:34.002244 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998254 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:18:34.002244 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998258 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:18:34.002244 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998260 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:18:34.002712 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998263 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:18:34.002712 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998266 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:18:34.002712 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998269 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:18:34.002712 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998272 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:18:34.002712 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998274 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:18:34.002712 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998277 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:18:34.002712 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998279 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:18:34.002712 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998281 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:18:34.002712 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998284 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:18:34.002712 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998286 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:18:34.002712 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998289 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:18:34.002712 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998291 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:18:34.002712 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998295 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:18:34.002712 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998298 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:18:34.002712 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998301 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:18:34.002712 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998304 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:18:34.002712 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998306 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:18:34.002712 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998309 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:18:34.002712 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998312 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:18:34.003203 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998315 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:18:34.003203 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998317 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:18:34.003203 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998320 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:18:34.003203 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998323 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:18:34.003203 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998325 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:18:34.003203 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998328 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:18:34.003203 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998332 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:18:34.003203 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998335 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:18:34.003203 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998337 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:18:34.003203 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998340 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:18:34.003203 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998344 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:18:34.003203 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998346 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:18:34.003203 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998348 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:18:34.003203 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998351 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:18:34.003203 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998353 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:18:34.003203 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998356 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:18:34.003203 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998358 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:18:34.003203 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998361 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:18:34.003203 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998363 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:18:34.003203 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998366 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:18:34.003679 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998368 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:18:34.003679 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998371 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:18:34.003679 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998373 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:18:34.003679 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998376 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:18:34.003679 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998378 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:18:34.003679 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998381 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:18:34.003679 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998384 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:18:34.003679 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998386 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:18:34.003679 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998389 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:18:34.003679 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998392 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:18:34.003679 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998394 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:18:34.003679 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998397 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:18:34.003679 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998399 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:18:34.003679 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998402 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:18:34.003679 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998404 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:18:34.003679 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998407 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:18:34.003679 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998409 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:18:34.003679 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998411 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:18:34.003679 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998414 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:18:34.003679 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998417 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:18:34.004186 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998419 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:18:34.004186 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998422 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:18:34.004186 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998424 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:18:34.004186 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998427 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:18:34.004186 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998429 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:18:34.004186 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998432 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:18:34.004186 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998434 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:18:34.004186 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:33.998437 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:18:34.004186 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.998442 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 05 13:18:34.004186 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:33.998579 2567 server.go:962] "Client rotation is on, will bootstrap in background" Jul 05 13:18:34.004186 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.001237 2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 05 13:18:34.004186 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.002967 2567 server.go:1019] "Starting client certificate rotation" Jul 05 13:18:34.004186 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.003074 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 05 13:18:34.004186 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.003105 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 05 13:18:34.006070 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.006053 2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 05 13:18:34.007596 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.007579 2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 05 13:18:34.021664 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.021641 2567 log.go:25] "Validated CRI v1 runtime API" Jul 05 13:18:34.024245 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.024216 2567 log.go:25] "Validated CRI v1 image API" Jul 05 13:18:34.025030 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.025016 2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 05 13:18:34.027779 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.027760 2567 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a360e0de-2bc0-46de-8592-e0710758b5e7:/dev/nvme0n1p3 ccc05e60-9e87-42cb-a61f-61287b4c5a5f:/dev/nvme0n1p4] Jul 05 13:18:34.027821 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.027779 2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 05 13:18:34.033185 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.033166 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 05 13:18:34.034021 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.033899 2567 manager.go:216] Machine: {Timestamp:2026-07-05 13:18:34.031426253 +0000 UTC m=+0.372444016 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100243 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29446b96f06775b54bc94232e65c2f SystemUUID:ec29446b-96f0-6775-b54b-c94232e65c2f BootID:5c414aea-8b34-4a7b-9d2c-67f03909e321 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:39:12:d8:2d:bb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:39:12:d8:2d:bb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:32:ae:66:83:ca:ba Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 05 13:18:34.034021 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.034020 2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 05 13:18:34.034127 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.034115 2567 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 05 13:18:34.036357 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.036327 2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 05 13:18:34.036505 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.036363 2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-119.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 05 13:18:34.037269 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.037257 2567 topology_manager.go:138] "Creating topology manager with none policy" Jul 05 13:18:34.037304 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.037271 2567 container_manager_linux.go:309] "Creating device plugin manager" Jul 05 13:18:34.037304 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.037279 2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 05 13:18:34.037304 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.037295 2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 05 13:18:34.039393 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.039379 2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 05 13:18:34.043581 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.043563 2567 state_mem.go:36] "Initialized new in-memory state store" Jul 05 13:18:34.043751 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.043727 2567 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 05 13:18:34.043791 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.043773 2567 kubelet.go:486] "Attempting to sync node with API server" Jul 05 13:18:34.043791 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.043786 2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 05 13:18:34.043845 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.043809 2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 05 13:18:34.043845 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.043821 2567 kubelet.go:398] "Adding apiserver pod source" Jul 05 13:18:34.043845 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.043832 2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 05 13:18:34.044280 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.044267 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 05 13:18:34.044327 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.044286 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 05 13:18:34.046929 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.046910 2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 05 13:18:34.049726 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.049712 2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 05 13:18:34.049789 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.049777 2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 05 13:18:34.051451 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.051440 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 05 13:18:34.051490 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.051456 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 05 13:18:34.051490 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.051463 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 05 13:18:34.051490 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.051468 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 05 13:18:34.051490 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.051476 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 05 13:18:34.051490 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.051482 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 05 13:18:34.051619 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.051493 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 05 13:18:34.051619 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.051501 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 05 13:18:34.051619 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.051507 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 05 13:18:34.051619 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.051512 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 05 13:18:34.051619 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.051529 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 05 13:18:34.051619 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.051534 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 05 13:18:34.053566 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.053555 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 05 13:18:34.053604 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.053567 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 05 13:18:34.054431 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:34.054407 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 05 13:18:34.054488 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:34.054411 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-119.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 05 13:18:34.056378 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.056362 2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-119.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 05 13:18:34.056959 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.056947 2567 server.go:1268] "Started kubelet" Jul 05 13:18:34.057814 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.057736 2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 05 13:18:34.057994 ip-10-0-141-119 systemd[1]: Started Kubernetes Kubelet. Jul 05 13:18:34.061023 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.060830 2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 05 13:18:34.061119 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.061062 2567 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 05 13:18:34.062497 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.062477 2567 server.go:310] "Adding debug handlers to kubelet server" Jul 05 13:18:34.063117 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.063100 2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 05 13:18:34.063336 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:34.063305 2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 05 13:18:34.068800 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:34.067690 2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-119.ec2.internal.18bf6742cc689c3b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-119.ec2.internal,UID:ip-10-0-141-119.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-119.ec2.internal,},FirstTimestamp:2026-07-05 13:18:34.056916027 +0000 UTC m=+0.397933790,LastTimestamp:2026-07-05 13:18:34.056916027 +0000 UTC m=+0.397933790,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-119.ec2.internal,}" Jul 05 13:18:34.069849 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.069836 2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 05 13:18:34.069849 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.069842 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 05 13:18:34.070030 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:34.069926 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-119.ec2.internal\" not found" Jul 05 13:18:34.070568 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.070543 2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 05 13:18:34.070568 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.070547 2567 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 05 13:18:34.070568 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.070571 2567 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 05 13:18:34.070870 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.070704 2567 reconstruct.go:97] "Volume reconstruction finished" Jul 05 13:18:34.070870 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.070871 2567 reconciler.go:29] "Reconciler: start to sync state" Jul 05 13:18:34.071028 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.070832 2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 05 13:18:34.071028 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.070893 2567 factory.go:55] Registering systemd factory Jul 05 13:18:34.071028 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.070906 2567 factory.go:223] Registration of the systemd container factory successfully Jul 05 13:18:34.071183 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.071167 2567 factory.go:153] Registering CRI-O factory Jul 05 13:18:34.071237 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.071188 2567 factory.go:223] Registration of the crio container factory successfully Jul 05 13:18:34.071237 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.071214 2567 factory.go:103] Registering Raw factory Jul 05 13:18:34.071237 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.071227 2567 manager.go:1196] Started watching for new ooms in manager Jul 05 13:18:34.071920 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.071903 2567 manager.go:349] Starting recovery of all containers Jul 05 13:18:34.076134 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:34.073466 2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-119.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 05 13:18:34.076134 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:34.073580 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 05 13:18:34.080550 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.080419 2567 manager.go:354] Recovery completed Jul 05 13:18:34.085263 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.085244 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:18:34.087272 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.087253 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-119.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:18:34.087345 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.087291 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-119.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:18:34.087345 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.087310 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-119.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:18:34.087904 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.087890 2567 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 05 13:18:34.087904 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.087903 2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 05 13:18:34.088008 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.087918 2567 state_mem.go:36] "Initialized new in-memory state store" Jul 05 13:18:34.088008 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.087931 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:18:34.088866 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.088849 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-119.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:18:34.088964 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.088889 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-119.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:18:34.088964 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.088904 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-119.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:18:34.089244 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.089232 2567 policy_none.go:49] "None policy: Start" Jul 05 13:18:34.089284 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.089248 2567 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 05 13:18:34.089284 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.089259 2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 05 13:18:34.089831 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:34.089725 2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-119.ec2.internal.18bf6742ce37d9a7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-119.ec2.internal,UID:ip-10-0-141-119.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-119.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-119.ec2.internal,},FirstTimestamp:2026-07-05 13:18:34.087274919 +0000 UTC m=+0.428292686,LastTimestamp:2026-07-05 13:18:34.087274919 +0000 UTC m=+0.428292686,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-119.ec2.internal,}" Jul 05 13:18:34.090422 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.090410 2567 policy_none.go:47] "Start" Jul 05 13:18:34.099297 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.099275 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zwmqz" Jul 05 13:18:34.108164 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:34.106771 2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-119.ec2.internal.18bf6742ce382ccc default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-119.ec2.internal,UID:ip-10-0-141-119.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-119.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-119.ec2.internal,},FirstTimestamp:2026-07-05 13:18:34.087296204 +0000 UTC m=+0.428313970,LastTimestamp:2026-07-05 13:18:34.087296204 +0000 UTC m=+0.428313970,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-119.ec2.internal,}" Jul 05 13:18:34.108164 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.107579 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zwmqz" Jul 05 13:18:34.126494 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.126476 2567 manager.go:337] "Starting Device Plugin manager" Jul 05 13:18:34.141927 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:34.126861 2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 05 13:18:34.141927 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.126884 2567 server.go:87] "Starting device plugin registration server" Jul 05 13:18:34.141927 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.127139 2567 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 05 13:18:34.141927 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.127151 2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 05 13:18:34.141927 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.127238 2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 05 13:18:34.141927 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.127307 2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 05 13:18:34.141927 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.127316 2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 05 13:18:34.141927 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:34.128017 2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 05 13:18:34.141927 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:34.128072 2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-119.ec2.internal\" not found" Jul 05 13:18:34.207447 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.207402 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 05 13:18:34.208798 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.208774 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 05 13:18:34.208919 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.208816 2567 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 05 13:18:34.208919 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.208839 2567 kubelet.go:2448] "Starting kubelet main sync loop" Jul 05 13:18:34.208919 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:34.208874 2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 05 13:18:34.211239 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.211223 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:18:34.228185 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.228103 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:18:34.229252 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.229233 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-119.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:18:34.229367 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.229264 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-119.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:18:34.229367 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.229274 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-119.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:18:34.229367 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.229297 2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-119.ec2.internal" Jul 05 13:18:34.238834 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.238804 2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-119.ec2.internal" Jul 05 13:18:34.238947 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:34.238837 2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-119.ec2.internal\": node \"ip-10-0-141-119.ec2.internal\" not found" Jul 05 13:18:34.265428 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:34.265395 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-119.ec2.internal\" not found" Jul 05 13:18:34.309452 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.309415 2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-119.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-119.ec2.internal"] Jul 05 13:18:34.309643 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.309508 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:18:34.310475 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.310453 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-119.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:18:34.310612 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.310491 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-119.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:18:34.310612 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.310507 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-119.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:18:34.311899 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.311885 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:18:34.312051 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.312017 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-119.ec2.internal" Jul 05 13:18:34.312116 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.312054 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:18:34.312669 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.312650 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-119.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:18:34.312795 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.312684 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-119.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:18:34.312795 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.312697 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-119.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:18:34.312795 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.312651 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-119.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:18:34.312795 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.312770 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-119.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:18:34.312795 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.312783 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-119.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:18:34.313949 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.313930 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-119.ec2.internal" Jul 05 13:18:34.313949 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.313959 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:18:34.314636 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.314605 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-119.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:18:34.314636 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.314626 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-119.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:18:34.314636 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.314635 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-119.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:18:34.329452 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:34.329422 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-119.ec2.internal\" not found" node="ip-10-0-141-119.ec2.internal" Jul 05 13:18:34.333033 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:34.333016 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-119.ec2.internal\" not found" node="ip-10-0-141-119.ec2.internal" Jul 05 13:18:34.366371 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:34.366335 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-119.ec2.internal\" not found" Jul 05 13:18:34.372719 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.372690 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7a0d072d628a3c33122d4e7cac8d89f1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-119.ec2.internal\" (UID: \"7a0d072d628a3c33122d4e7cac8d89f1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-119.ec2.internal" Jul 05 13:18:34.372825 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.372723 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a0d072d628a3c33122d4e7cac8d89f1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-119.ec2.internal\" (UID: \"7a0d072d628a3c33122d4e7cac8d89f1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-119.ec2.internal" Jul 05 13:18:34.372825 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.372767 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a1a32c2f892e1c59cdaebdb361cae096-config\") pod \"kube-apiserver-proxy-ip-10-0-141-119.ec2.internal\" (UID: \"a1a32c2f892e1c59cdaebdb361cae096\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-119.ec2.internal" Jul 05 13:18:34.466669 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:34.466634 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-119.ec2.internal\" not found" Jul 05 13:18:34.473022 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.472983 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7a0d072d628a3c33122d4e7cac8d89f1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-119.ec2.internal\" (UID: \"7a0d072d628a3c33122d4e7cac8d89f1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-119.ec2.internal" Jul 05 13:18:34.473152 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.473061 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7a0d072d628a3c33122d4e7cac8d89f1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-119.ec2.internal\" (UID: \"7a0d072d628a3c33122d4e7cac8d89f1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-119.ec2.internal" Jul 05 13:18:34.473152 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.473078 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a0d072d628a3c33122d4e7cac8d89f1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-119.ec2.internal\" (UID: \"7a0d072d628a3c33122d4e7cac8d89f1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-119.ec2.internal" Jul 05 13:18:34.473152 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.473102 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a1a32c2f892e1c59cdaebdb361cae096-config\") pod \"kube-apiserver-proxy-ip-10-0-141-119.ec2.internal\" (UID: \"a1a32c2f892e1c59cdaebdb361cae096\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-119.ec2.internal" Jul 05 13:18:34.473152 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.473143 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a0d072d628a3c33122d4e7cac8d89f1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-119.ec2.internal\" (UID: \"7a0d072d628a3c33122d4e7cac8d89f1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-119.ec2.internal" Jul 05 13:18:34.473291 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.473151 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a1a32c2f892e1c59cdaebdb361cae096-config\") pod \"kube-apiserver-proxy-ip-10-0-141-119.ec2.internal\" (UID: \"a1a32c2f892e1c59cdaebdb361cae096\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-119.ec2.internal" Jul 05 13:18:34.567484 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:34.567404 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-119.ec2.internal\" not found" Jul 05 13:18:34.632994 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.632950 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-119.ec2.internal" Jul 05 13:18:34.637281 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:34.637132 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-119.ec2.internal" Jul 05 13:18:34.668307 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:34.668272 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-119.ec2.internal\" not found" Jul 05 13:18:34.768904 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:34.768867 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-119.ec2.internal\" not found" Jul 05 13:18:34.869453 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:34.869370 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-119.ec2.internal\" not found" Jul 05 13:18:34.969941 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:34.969905 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-119.ec2.internal\" not found" Jul 05 13:18:35.002407 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:35.002370 2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 05 13:18:35.003017 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:35.002512 2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 05 13:18:35.070049 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:35.070010 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-119.ec2.internal\" not found" Jul 05 13:18:35.113360 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:35.113295 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-04 13:13:34 +0000 UTC" deadline="2028-04-21 11:04:41.559697 +0000 UTC" Jul 05 13:18:35.113360 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:35.113348 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15741h46m6.44635324s" Jul 05 13:18:35.170660 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:35.170628 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-119.ec2.internal\" not found" Jul 05 13:18:35.223926 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:35.223885 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:18:35.224975 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:35.224957 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-119.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:18:35.225087 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:35.224991 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-119.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:18:35.225087 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:35.225000 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-119.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:18:35.233228 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:35.233205 2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:18:35.241713 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:35.241672 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a0d072d628a3c33122d4e7cac8d89f1.slice/crio-a073fe1ee43ec1ed613b4d8db25671a4d3e1b37399718653b54a54d097473f6f WatchSource:0}: Error finding container a073fe1ee43ec1ed613b4d8db25671a4d3e1b37399718653b54a54d097473f6f: Status 404 returned error can't find the container with id a073fe1ee43ec1ed613b4d8db25671a4d3e1b37399718653b54a54d097473f6f Jul 05 13:18:35.242116 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:35.242090 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1a32c2f892e1c59cdaebdb361cae096.slice/crio-2b86c4900594d5020651d115e941605ef0448f70d1c69545a28e0e7b9288c7b0 WatchSource:0}: Error finding container 2b86c4900594d5020651d115e941605ef0448f70d1c69545a28e0e7b9288c7b0: Status 404 returned error can't find the container with id 2b86c4900594d5020651d115e941605ef0448f70d1c69545a28e0e7b9288c7b0 Jul 05 13:18:35.247206 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:35.247186 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 13:18:35.270569 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:35.270531 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-119.ec2.internal" Jul 05 13:18:35.284056 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:35.284029 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 05 13:18:35.285811 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:35.285796 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-119.ec2.internal" Jul 05 13:18:35.295602 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:35.295579 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 05 13:18:35.372088 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:35.372051 2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:18:35.386598 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:35.386524 2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:18:35.883492 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:35.883254 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:18:36.045342 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.045276 2567 apiserver.go:52] "Watching apiserver" Jul 05 13:18:36.055029 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.054990 2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 05 13:18:36.056200 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.056173 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-9sgqr","openshift-network-diagnostics/network-check-target-j5rvd","openshift-network-operator/iptables-alerter-xj828","openshift-ovn-kubernetes/ovnkube-node-swckn","kube-system/global-pull-secret-syncer-k92vt","kube-system/kube-apiserver-proxy-ip-10-0-141-119.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw","openshift-cluster-node-tuning-operator/tuned-mhlfp","openshift-image-registry/node-ca-8s4z9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-119.ec2.internal","openshift-multus/multus-additional-cni-plugins-wk7fc","openshift-multus/multus-k882t","kube-system/konnectivity-agent-qmf78","openshift-dns/node-resolver-78f24"] Jul 05 13:18:36.057597 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.057570 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" Jul 05 13:18:36.058930 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.058904 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qmf78" Jul 05 13:18:36.060226 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.060200 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xj828" Jul 05 13:18:36.061659 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.061635 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.063781 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.063508 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:18:36.063781 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:36.063617 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k92vt" podUID="32a4a45d-3b64-476e-9147-09693f809841" Jul 05 13:18:36.063925 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.063890 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 05 13:18:36.064088 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.064068 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 05 13:18:36.064322 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.064300 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 05 13:18:36.065288 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.065263 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 05 13:18:36.065508 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.065487 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:18:36.068921 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.068891 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 05 13:18:36.069165 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.069147 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cb786\"" Jul 05 13:18:36.069370 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.069353 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-g9p7x\"" Jul 05 13:18:36.069584 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.069561 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:18:36.069654 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:36.069633 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sgqr" podUID="bcbf773f-ee0a-42bd-8983-a1a3bb066e5d" Jul 05 13:18:36.069712 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.069696 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-78f24" Jul 05 13:18:36.070112 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.070088 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8s4z9" Jul 05 13:18:36.070265 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.070192 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 05 13:18:36.070943 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.070925 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 05 13:18:36.071016 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.070951 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 05 13:18:36.071016 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.070996 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 05 13:18:36.071141 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.071050 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hgxsb\"" Jul 05 13:18:36.071234 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.071218 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:18:36.071300 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:36.071280 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j5rvd" podUID="2d509ded-4089-44e1-994f-75d2b4570fce" Jul 05 13:18:36.071358 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.071309 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 05 13:18:36.071408 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.071364 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 05 13:18:36.071563 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.071547 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 05 13:18:36.071699 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.071685 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.071699 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.071695 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 05 13:18:36.071858 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.071844 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rw2g6\"" Jul 05 13:18:36.071995 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.071984 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 05 13:18:36.072243 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.072229 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 05 13:18:36.072535 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.072512 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 05 13:18:36.072935 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.072912 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-smh84\"" Jul 05 13:18:36.073347 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.073325 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 05 13:18:36.073443 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.073357 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 05 13:18:36.074015 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.073966 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.074274 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.074253 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rngph\"" Jul 05 13:18:36.075182 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.074879 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k882t" Jul 05 13:18:36.075182 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.075171 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 05 13:18:36.076698 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.076675 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-m8hgn\"" Jul 05 13:18:36.076954 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.076935 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 05 13:18:36.077523 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.077501 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:18:36.077871 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.077848 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 05 13:18:36.079079 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.078207 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ckmtn\"" Jul 05 13:18:36.080353 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.080312 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 05 13:18:36.080932 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.080911 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 05 13:18:36.081114 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.081094 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-gkhbr\"" Jul 05 13:18:36.081471 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.081441 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 05 13:18:36.081639 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.081623 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 05 13:18:36.081804 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.081669 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 05 13:18:36.082366 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082343 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fed1aa39-8ecb-4770-9dea-c73e7b76386a-registration-dir\") pod \"aws-ebs-csi-driver-node-gkcrw\" (UID: \"fed1aa39-8ecb-4770-9dea-c73e7b76386a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" Jul 05 13:18:36.082448 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082381 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/df0a5b01-f000-4879-8fcc-175f435bce3e-agent-certs\") pod \"konnectivity-agent-qmf78\" (UID: \"df0a5b01-f000-4879-8fcc-175f435bce3e\") " pod="kube-system/konnectivity-agent-qmf78" Jul 05 13:18:36.082448 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082407 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-metrics-certs\") pod \"network-metrics-daemon-9sgqr\" (UID: \"bcbf773f-ee0a-42bd-8983-a1a3bb066e5d\") " pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:18:36.082448 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082432 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-host-kubelet\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.082646 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082450 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a447ce81-b3cb-4269-8d46-e6e612b89e0b-ovnkube-script-lib\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.082646 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082468 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fed1aa39-8ecb-4770-9dea-c73e7b76386a-sys-fs\") pod \"aws-ebs-csi-driver-node-gkcrw\" (UID: \"fed1aa39-8ecb-4770-9dea-c73e7b76386a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" Jul 05 13:18:36.082646 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082483 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7pc4\" (UniqueName: \"kubernetes.io/projected/8af4784e-62fd-4a92-ba60-c91f69a2322b-kube-api-access-s7pc4\") pod \"node-resolver-78f24\" (UID: \"8af4784e-62fd-4a92-ba60-c91f69a2322b\") " pod="openshift-dns/node-resolver-78f24" Jul 05 13:18:36.082646 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082497 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-etc-systemd\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.082646 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082513 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-node-log\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.082646 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082528 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fed1aa39-8ecb-4770-9dea-c73e7b76386a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gkcrw\" (UID: \"fed1aa39-8ecb-4770-9dea-c73e7b76386a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" Jul 05 13:18:36.082646 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082541 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/84b36d5b-eb96-4252-a0d2-a1f5d8c58a4e-serviceca\") pod \"node-ca-8s4z9\" (UID: \"84b36d5b-eb96-4252-a0d2-a1f5d8c58a4e\") " pod="openshift-image-registry/node-ca-8s4z9" Jul 05 13:18:36.082646 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082557 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-var-lib-kubelet\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.082646 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082571 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-run-ovn\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.082646 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082586 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d164f387-3a93-465a-a587-ac48c676e2f6-cnibin\") pod \"multus-additional-cni-plugins-wk7fc\" (UID: \"d164f387-3a93-465a-a587-ac48c676e2f6\") " pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.082646 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082601 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d164f387-3a93-465a-a587-ac48c676e2f6-os-release\") pod \"multus-additional-cni-plugins-wk7fc\" (UID: \"d164f387-3a93-465a-a587-ac48c676e2f6\") " pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.082646 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082615 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fed1aa39-8ecb-4770-9dea-c73e7b76386a-socket-dir\") pod \"aws-ebs-csi-driver-node-gkcrw\" (UID: \"fed1aa39-8ecb-4770-9dea-c73e7b76386a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" Jul 05 13:18:36.082646 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082630 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-etc-sysconfig\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.082646 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082647 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-run-systemd\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.083160 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082662 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32a4a45d-3b64-476e-9147-09693f809841-original-pull-secret\") pod \"global-pull-secret-syncer-k92vt\" (UID: \"32a4a45d-3b64-476e-9147-09693f809841\") " pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:18:36.083160 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082744 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrttb\" (UniqueName: \"kubernetes.io/projected/cd1a8674-00ab-40d3-b169-f5857832053a-kube-api-access-nrttb\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.083160 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082759 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a447ce81-b3cb-4269-8d46-e6e612b89e0b-ovnkube-config\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.083160 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082832 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-lib-modules\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.083160 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082848 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-host\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.083160 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082862 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-var-lib-openvswitch\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.083160 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082883 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-log-socket\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.083160 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082899 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-host-run-ovn-kubernetes\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.083160 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082914 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7364f530-c4ff-47ba-a86b-904542a26d07-host-slash\") pod \"iptables-alerter-xj828\" (UID: \"7364f530-c4ff-47ba-a86b-904542a26d07\") " pod="openshift-network-operator/iptables-alerter-xj828" Jul 05 13:18:36.083160 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082931 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d164f387-3a93-465a-a587-ac48c676e2f6-cni-binary-copy\") pod \"multus-additional-cni-plugins-wk7fc\" (UID: \"d164f387-3a93-465a-a587-ac48c676e2f6\") " pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.083160 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082946 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d164f387-3a93-465a-a587-ac48c676e2f6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wk7fc\" (UID: \"d164f387-3a93-465a-a587-ac48c676e2f6\") " pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.083160 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082962 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/84b36d5b-eb96-4252-a0d2-a1f5d8c58a4e-host\") pod \"node-ca-8s4z9\" (UID: \"84b36d5b-eb96-4252-a0d2-a1f5d8c58a4e\") " pod="openshift-image-registry/node-ca-8s4z9" Jul 05 13:18:36.083160 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082977 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d458z\" (UniqueName: \"kubernetes.io/projected/a447ce81-b3cb-4269-8d46-e6e612b89e0b-kube-api-access-d458z\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.083160 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.082995 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tngz8\" (UniqueName: \"kubernetes.io/projected/fed1aa39-8ecb-4770-9dea-c73e7b76386a-kube-api-access-tngz8\") pod \"aws-ebs-csi-driver-node-gkcrw\" (UID: \"fed1aa39-8ecb-4770-9dea-c73e7b76386a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" Jul 05 13:18:36.083160 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083016 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8af4784e-62fd-4a92-ba60-c91f69a2322b-hosts-file\") pod \"node-resolver-78f24\" (UID: \"8af4784e-62fd-4a92-ba60-c91f69a2322b\") " pod="openshift-dns/node-resolver-78f24" Jul 05 13:18:36.083160 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083034 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8af4784e-62fd-4a92-ba60-c91f69a2322b-tmp-dir\") pod \"node-resolver-78f24\" (UID: \"8af4784e-62fd-4a92-ba60-c91f69a2322b\") " pod="openshift-dns/node-resolver-78f24" Jul 05 13:18:36.083832 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083051 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpbxg\" (UniqueName: \"kubernetes.io/projected/84b36d5b-eb96-4252-a0d2-a1f5d8c58a4e-kube-api-access-xpbxg\") pod \"node-ca-8s4z9\" (UID: \"84b36d5b-eb96-4252-a0d2-a1f5d8c58a4e\") " pod="openshift-image-registry/node-ca-8s4z9" Jul 05 13:18:36.083832 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083067 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6w76\" (UniqueName: \"kubernetes.io/projected/2d509ded-4089-44e1-994f-75d2b4570fce-kube-api-access-x6w76\") pod \"network-check-target-j5rvd\" (UID: \"2d509ded-4089-44e1-994f-75d2b4570fce\") " pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:18:36.083832 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083082 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-etc-kubernetes\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.083832 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083107 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d164f387-3a93-465a-a587-ac48c676e2f6-system-cni-dir\") pod \"multus-additional-cni-plugins-wk7fc\" (UID: \"d164f387-3a93-465a-a587-ac48c676e2f6\") " pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.083832 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083137 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/32a4a45d-3b64-476e-9147-09693f809841-kubelet-config\") pod \"global-pull-secret-syncer-k92vt\" (UID: \"32a4a45d-3b64-476e-9147-09693f809841\") " pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:18:36.083832 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083363 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/32a4a45d-3b64-476e-9147-09693f809841-dbus\") pod \"global-pull-secret-syncer-k92vt\" (UID: \"32a4a45d-3b64-476e-9147-09693f809841\") " pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:18:36.083832 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083410 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cd1a8674-00ab-40d3-b169-f5857832053a-etc-tuned\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.083832 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083453 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd1a8674-00ab-40d3-b169-f5857832053a-tmp\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.083832 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083567 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7364f530-c4ff-47ba-a86b-904542a26d07-iptables-alerter-script\") pod \"iptables-alerter-xj828\" (UID: \"7364f530-c4ff-47ba-a86b-904542a26d07\") " pod="openshift-network-operator/iptables-alerter-xj828" Jul 05 13:18:36.083832 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083601 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-host-cni-netd\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.083832 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083635 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.083832 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083655 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fed1aa39-8ecb-4770-9dea-c73e7b76386a-etc-selinux\") pod \"aws-ebs-csi-driver-node-gkcrw\" (UID: \"fed1aa39-8ecb-4770-9dea-c73e7b76386a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" Jul 05 13:18:36.083832 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083674 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvcfs\" (UniqueName: \"kubernetes.io/projected/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-kube-api-access-qvcfs\") pod \"network-metrics-daemon-9sgqr\" (UID: \"bcbf773f-ee0a-42bd-8983-a1a3bb066e5d\") " pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:18:36.083832 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083699 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-host-slash\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.083832 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083740 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-etc-openvswitch\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.083832 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083799 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-run-openvswitch\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.084405 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083816 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-etc-sysctl-conf\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.084405 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083832 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-run\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.084405 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083851 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-sys\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.084405 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083874 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-host-cni-bin\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.084405 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083897 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d164f387-3a93-465a-a587-ac48c676e2f6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wk7fc\" (UID: \"d164f387-3a93-465a-a587-ac48c676e2f6\") " pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.084405 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083914 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snv5l\" (UniqueName: \"kubernetes.io/projected/d164f387-3a93-465a-a587-ac48c676e2f6-kube-api-access-snv5l\") pod \"multus-additional-cni-plugins-wk7fc\" (UID: \"d164f387-3a93-465a-a587-ac48c676e2f6\") " pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.084405 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083934 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbwhx\" (UniqueName: \"kubernetes.io/projected/7364f530-c4ff-47ba-a86b-904542a26d07-kube-api-access-wbwhx\") pod \"iptables-alerter-xj828\" (UID: \"7364f530-c4ff-47ba-a86b-904542a26d07\") " pod="openshift-network-operator/iptables-alerter-xj828" Jul 05 13:18:36.084405 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083960 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-systemd-units\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.084405 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.083984 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-host-run-netns\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.084405 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.084001 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a447ce81-b3cb-4269-8d46-e6e612b89e0b-env-overrides\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.084405 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.084017 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d164f387-3a93-465a-a587-ac48c676e2f6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wk7fc\" (UID: \"d164f387-3a93-465a-a587-ac48c676e2f6\") " pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.084405 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.084032 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fed1aa39-8ecb-4770-9dea-c73e7b76386a-device-dir\") pod \"aws-ebs-csi-driver-node-gkcrw\" (UID: \"fed1aa39-8ecb-4770-9dea-c73e7b76386a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" Jul 05 13:18:36.084405 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.084053 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a447ce81-b3cb-4269-8d46-e6e612b89e0b-ovn-node-metrics-cert\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.084405 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.084094 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/df0a5b01-f000-4879-8fcc-175f435bce3e-konnectivity-ca\") pod \"konnectivity-agent-qmf78\" (UID: \"df0a5b01-f000-4879-8fcc-175f435bce3e\") " pod="kube-system/konnectivity-agent-qmf78" Jul 05 13:18:36.084405 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.084131 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-etc-modprobe-d\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.084405 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.084179 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-etc-sysctl-d\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.084933 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.084326 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 05 13:18:36.122101 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.122070 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vth2r" Jul 05 13:18:36.130951 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.130922 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vth2r" Jul 05 13:18:36.171987 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.171956 2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 05 13:18:36.185295 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.185246 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-multus-socket-dir-parent\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.185295 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.185301 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-host-run-netns\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.185538 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.185336 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-etc-sysctl-conf\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.185538 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.185387 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-run\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.185538 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.185436 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-sys\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.185538 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.185497 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-etc-sysctl-conf\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.185538 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.185527 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-run\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.185538 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.185536 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-host-cni-bin\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.185868 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.185527 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-sys\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.185868 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.185573 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d164f387-3a93-465a-a587-ac48c676e2f6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wk7fc\" (UID: \"d164f387-3a93-465a-a587-ac48c676e2f6\") " pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.185868 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.185603 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snv5l\" (UniqueName: \"kubernetes.io/projected/d164f387-3a93-465a-a587-ac48c676e2f6-kube-api-access-snv5l\") pod \"multus-additional-cni-plugins-wk7fc\" (UID: \"d164f387-3a93-465a-a587-ac48c676e2f6\") " pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.185868 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.185578 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-host-cni-bin\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.185868 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.185632 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-host-var-lib-cni-multus\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.185868 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.185666 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbwhx\" (UniqueName: \"kubernetes.io/projected/7364f530-c4ff-47ba-a86b-904542a26d07-kube-api-access-wbwhx\") pod \"iptables-alerter-xj828\" (UID: \"7364f530-c4ff-47ba-a86b-904542a26d07\") " pod="openshift-network-operator/iptables-alerter-xj828" Jul 05 13:18:36.185868 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.185705 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-systemd-units\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.185868 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.185772 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-systemd-units\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.185868 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.185855 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-host-run-netns\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.186227 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.185898 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-host-run-netns\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.186227 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.185909 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a447ce81-b3cb-4269-8d46-e6e612b89e0b-env-overrides\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.186227 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186085 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d164f387-3a93-465a-a587-ac48c676e2f6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wk7fc\" (UID: \"d164f387-3a93-465a-a587-ac48c676e2f6\") " pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.186227 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186138 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fed1aa39-8ecb-4770-9dea-c73e7b76386a-device-dir\") pod \"aws-ebs-csi-driver-node-gkcrw\" (UID: \"fed1aa39-8ecb-4770-9dea-c73e7b76386a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" Jul 05 13:18:36.186227 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186216 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a447ce81-b3cb-4269-8d46-e6e612b89e0b-ovn-node-metrics-cert\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.186490 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186253 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-cnibin\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.186490 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186280 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-host-run-multus-certs\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.186490 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186299 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d164f387-3a93-465a-a587-ac48c676e2f6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wk7fc\" (UID: \"d164f387-3a93-465a-a587-ac48c676e2f6\") " pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.186490 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186308 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/df0a5b01-f000-4879-8fcc-175f435bce3e-konnectivity-ca\") pod \"konnectivity-agent-qmf78\" (UID: \"df0a5b01-f000-4879-8fcc-175f435bce3e\") " pod="kube-system/konnectivity-agent-qmf78" Jul 05 13:18:36.186490 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186336 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-etc-modprobe-d\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.186490 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186372 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-etc-sysctl-d\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.186490 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186395 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-multus-cni-dir\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.186490 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186429 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fed1aa39-8ecb-4770-9dea-c73e7b76386a-registration-dir\") pod \"aws-ebs-csi-driver-node-gkcrw\" (UID: \"fed1aa39-8ecb-4770-9dea-c73e7b76386a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" Jul 05 13:18:36.186490 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186465 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a447ce81-b3cb-4269-8d46-e6e612b89e0b-env-overrides\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.186490 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186479 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/df0a5b01-f000-4879-8fcc-175f435bce3e-agent-certs\") pod \"konnectivity-agent-qmf78\" (UID: \"df0a5b01-f000-4879-8fcc-175f435bce3e\") " pod="kube-system/konnectivity-agent-qmf78" Jul 05 13:18:36.187018 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186517 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-metrics-certs\") pod \"network-metrics-daemon-9sgqr\" (UID: \"bcbf773f-ee0a-42bd-8983-a1a3bb066e5d\") " pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:18:36.187018 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186543 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-host-kubelet\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.187018 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186589 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a447ce81-b3cb-4269-8d46-e6e612b89e0b-ovnkube-script-lib\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.187018 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186623 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f2d4dd11-346b-496e-886b-9238eaa91508-multus-daemon-config\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.187018 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186649 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fed1aa39-8ecb-4770-9dea-c73e7b76386a-sys-fs\") pod \"aws-ebs-csi-driver-node-gkcrw\" (UID: \"fed1aa39-8ecb-4770-9dea-c73e7b76386a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" Jul 05 13:18:36.187018 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186688 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7pc4\" (UniqueName: \"kubernetes.io/projected/8af4784e-62fd-4a92-ba60-c91f69a2322b-kube-api-access-s7pc4\") pod \"node-resolver-78f24\" (UID: \"8af4784e-62fd-4a92-ba60-c91f69a2322b\") " pod="openshift-dns/node-resolver-78f24" Jul 05 13:18:36.187018 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186701 2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 05 13:18:36.187018 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186718 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-etc-systemd\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.187018 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186771 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-node-log\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.187018 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186783 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fed1aa39-8ecb-4770-9dea-c73e7b76386a-registration-dir\") pod \"aws-ebs-csi-driver-node-gkcrw\" (UID: \"fed1aa39-8ecb-4770-9dea-c73e7b76386a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" Jul 05 13:18:36.187018 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186832 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g688n\" (UniqueName: \"kubernetes.io/projected/f2d4dd11-346b-496e-886b-9238eaa91508-kube-api-access-g688n\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.187018 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186898 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fed1aa39-8ecb-4770-9dea-c73e7b76386a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gkcrw\" (UID: \"fed1aa39-8ecb-4770-9dea-c73e7b76386a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" Jul 05 13:18:36.187018 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186933 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/84b36d5b-eb96-4252-a0d2-a1f5d8c58a4e-serviceca\") pod \"node-ca-8s4z9\" (UID: \"84b36d5b-eb96-4252-a0d2-a1f5d8c58a4e\") " pod="openshift-image-registry/node-ca-8s4z9" Jul 05 13:18:36.187018 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186959 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-var-lib-kubelet\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.187018 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186982 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-run-ovn\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.187018 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.186980 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d164f387-3a93-465a-a587-ac48c676e2f6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wk7fc\" (UID: \"d164f387-3a93-465a-a587-ac48c676e2f6\") " pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.187018 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.187012 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d164f387-3a93-465a-a587-ac48c676e2f6-cnibin\") pod \"multus-additional-cni-plugins-wk7fc\" (UID: \"d164f387-3a93-465a-a587-ac48c676e2f6\") " pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.187572 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.187063 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d164f387-3a93-465a-a587-ac48c676e2f6-os-release\") pod \"multus-additional-cni-plugins-wk7fc\" (UID: \"d164f387-3a93-465a-a587-ac48c676e2f6\") " pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.187572 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.187088 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-host-var-lib-kubelet\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.187572 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.187112 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-hostroot\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.187572 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.187143 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fed1aa39-8ecb-4770-9dea-c73e7b76386a-socket-dir\") pod \"aws-ebs-csi-driver-node-gkcrw\" (UID: \"fed1aa39-8ecb-4770-9dea-c73e7b76386a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" Jul 05 13:18:36.187572 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.187191 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-etc-sysconfig\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.187572 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.187220 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-run-systemd\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.187572 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.187319 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/df0a5b01-f000-4879-8fcc-175f435bce3e-konnectivity-ca\") pod \"konnectivity-agent-qmf78\" (UID: \"df0a5b01-f000-4879-8fcc-175f435bce3e\") " pod="kube-system/konnectivity-agent-qmf78" Jul 05 13:18:36.187572 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.187396 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-host-run-k8s-cni-cncf-io\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.187572 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.187428 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-etc-modprobe-d\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.187572 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.187485 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32a4a45d-3b64-476e-9147-09693f809841-original-pull-secret\") pod \"global-pull-secret-syncer-k92vt\" (UID: \"32a4a45d-3b64-476e-9147-09693f809841\") " pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:18:36.187572 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.187530 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrttb\" (UniqueName: \"kubernetes.io/projected/cd1a8674-00ab-40d3-b169-f5857832053a-kube-api-access-nrttb\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.187572 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.187561 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-etc-sysctl-d\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.187572 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.187559 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a447ce81-b3cb-4269-8d46-e6e612b89e0b-ovnkube-config\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.188007 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:36.187989 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:36.188102 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:36.188083 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-metrics-certs podName:bcbf773f-ee0a-42bd-8983-a1a3bb066e5d nodeName:}" failed. No retries permitted until 2026-07-05 13:18:36.688062987 +0000 UTC m=+3.029080751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-metrics-certs") pod "network-metrics-daemon-9sgqr" (UID: "bcbf773f-ee0a-42bd-8983-a1a3bb066e5d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:36.188216 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.188106 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a447ce81-b3cb-4269-8d46-e6e612b89e0b-ovnkube-config\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.188216 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.188136 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-etc-systemd\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.188216 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.188179 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-var-lib-kubelet\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.188216 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.188189 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-node-log\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.188345 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.188251 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fed1aa39-8ecb-4770-9dea-c73e7b76386a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gkcrw\" (UID: \"fed1aa39-8ecb-4770-9dea-c73e7b76386a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" Jul 05 13:18:36.188345 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.188262 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-run-ovn\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.188345 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.188292 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-host-kubelet\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.188345 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.188329 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d164f387-3a93-465a-a587-ac48c676e2f6-cnibin\") pod \"multus-additional-cni-plugins-wk7fc\" (UID: \"d164f387-3a93-465a-a587-ac48c676e2f6\") " pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.188465 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.188455 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fed1aa39-8ecb-4770-9dea-c73e7b76386a-socket-dir\") pod \"aws-ebs-csi-driver-node-gkcrw\" (UID: \"fed1aa39-8ecb-4770-9dea-c73e7b76386a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" Jul 05 13:18:36.188543 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.188510 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-etc-sysconfig\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.188630 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.188600 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-run-systemd\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.188736 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:36.188721 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:36.188855 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.188838 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/84b36d5b-eb96-4252-a0d2-a1f5d8c58a4e-serviceca\") pod \"node-ca-8s4z9\" (UID: \"84b36d5b-eb96-4252-a0d2-a1f5d8c58a4e\") " pod="openshift-image-registry/node-ca-8s4z9" Jul 05 13:18:36.188901 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:36.188869 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32a4a45d-3b64-476e-9147-09693f809841-original-pull-secret podName:32a4a45d-3b64-476e-9147-09693f809841 nodeName:}" failed. No retries permitted until 2026-07-05 13:18:36.6888519 +0000 UTC m=+3.029869862 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/32a4a45d-3b64-476e-9147-09693f809841-original-pull-secret") pod "global-pull-secret-syncer-k92vt" (UID: "32a4a45d-3b64-476e-9147-09693f809841") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:36.188901 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.187499 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fed1aa39-8ecb-4770-9dea-c73e7b76386a-device-dir\") pod \"aws-ebs-csi-driver-node-gkcrw\" (UID: \"fed1aa39-8ecb-4770-9dea-c73e7b76386a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" Jul 05 13:18:36.188901 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.188894 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-lib-modules\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.189003 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.188921 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-host\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.189003 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.188936 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fed1aa39-8ecb-4770-9dea-c73e7b76386a-sys-fs\") pod \"aws-ebs-csi-driver-node-gkcrw\" (UID: \"fed1aa39-8ecb-4770-9dea-c73e7b76386a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" Jul 05 13:18:36.189003 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.188971 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-var-lib-openvswitch\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.189003 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.188995 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-log-socket\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.189121 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.189015 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a447ce81-b3cb-4269-8d46-e6e612b89e0b-ovnkube-script-lib\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.189121 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.189020 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d164f387-3a93-465a-a587-ac48c676e2f6-os-release\") pod \"multus-additional-cni-plugins-wk7fc\" (UID: \"d164f387-3a93-465a-a587-ac48c676e2f6\") " pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.189121 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.189020 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-host-run-ovn-kubernetes\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.189121 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.189086 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-host\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.189121 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.189087 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-log-socket\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.189311 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.189157 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-host-run-ovn-kubernetes\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.189311 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.189171 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-var-lib-openvswitch\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.189311 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.189222 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7364f530-c4ff-47ba-a86b-904542a26d07-host-slash\") pod \"iptables-alerter-xj828\" (UID: \"7364f530-c4ff-47ba-a86b-904542a26d07\") " pod="openshift-network-operator/iptables-alerter-xj828" Jul 05 13:18:36.189311 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.189260 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d164f387-3a93-465a-a587-ac48c676e2f6-cni-binary-copy\") pod \"multus-additional-cni-plugins-wk7fc\" (UID: \"d164f387-3a93-465a-a587-ac48c676e2f6\") " pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.189311 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.189296 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7364f530-c4ff-47ba-a86b-904542a26d07-host-slash\") pod \"iptables-alerter-xj828\" (UID: \"7364f530-c4ff-47ba-a86b-904542a26d07\") " pod="openshift-network-operator/iptables-alerter-xj828" Jul 05 13:18:36.189579 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.189265 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-lib-modules\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.189579 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.189346 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d164f387-3a93-465a-a587-ac48c676e2f6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wk7fc\" (UID: \"d164f387-3a93-465a-a587-ac48c676e2f6\") " pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.189579 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.189388 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-host-var-lib-cni-bin\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.189579 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.189489 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-etc-kubernetes\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.189579 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.189536 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/84b36d5b-eb96-4252-a0d2-a1f5d8c58a4e-host\") pod \"node-ca-8s4z9\" (UID: \"84b36d5b-eb96-4252-a0d2-a1f5d8c58a4e\") " pod="openshift-image-registry/node-ca-8s4z9" Jul 05 13:18:36.189579 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.189563 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d458z\" (UniqueName: \"kubernetes.io/projected/a447ce81-b3cb-4269-8d46-e6e612b89e0b-kube-api-access-d458z\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.189579 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.189570 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d164f387-3a93-465a-a587-ac48c676e2f6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wk7fc\" (UID: \"d164f387-3a93-465a-a587-ac48c676e2f6\") " pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.189911 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.189618 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/84b36d5b-eb96-4252-a0d2-a1f5d8c58a4e-host\") pod \"node-ca-8s4z9\" (UID: \"84b36d5b-eb96-4252-a0d2-a1f5d8c58a4e\") " pod="openshift-image-registry/node-ca-8s4z9" Jul 05 13:18:36.189957 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.189931 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d164f387-3a93-465a-a587-ac48c676e2f6-cni-binary-copy\") pod \"multus-additional-cni-plugins-wk7fc\" (UID: \"d164f387-3a93-465a-a587-ac48c676e2f6\") " pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.190626 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.190593 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-system-cni-dir\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.190745 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.190645 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2d4dd11-346b-496e-886b-9238eaa91508-cni-binary-copy\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.190745 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.190666 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-multus-conf-dir\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.190745 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.190710 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tngz8\" (UniqueName: \"kubernetes.io/projected/fed1aa39-8ecb-4770-9dea-c73e7b76386a-kube-api-access-tngz8\") pod \"aws-ebs-csi-driver-node-gkcrw\" (UID: \"fed1aa39-8ecb-4770-9dea-c73e7b76386a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" Jul 05 13:18:36.190936 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.190804 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8af4784e-62fd-4a92-ba60-c91f69a2322b-hosts-file\") pod \"node-resolver-78f24\" (UID: \"8af4784e-62fd-4a92-ba60-c91f69a2322b\") " pod="openshift-dns/node-resolver-78f24" Jul 05 13:18:36.190936 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.190849 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8af4784e-62fd-4a92-ba60-c91f69a2322b-tmp-dir\") pod \"node-resolver-78f24\" (UID: \"8af4784e-62fd-4a92-ba60-c91f69a2322b\") " pod="openshift-dns/node-resolver-78f24" Jul 05 13:18:36.190936 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.190918 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpbxg\" (UniqueName: \"kubernetes.io/projected/84b36d5b-eb96-4252-a0d2-a1f5d8c58a4e-kube-api-access-xpbxg\") pod \"node-ca-8s4z9\" (UID: \"84b36d5b-eb96-4252-a0d2-a1f5d8c58a4e\") " pod="openshift-image-registry/node-ca-8s4z9" Jul 05 13:18:36.191092 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.190961 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6w76\" (UniqueName: \"kubernetes.io/projected/2d509ded-4089-44e1-994f-75d2b4570fce-kube-api-access-x6w76\") pod \"network-check-target-j5rvd\" (UID: \"2d509ded-4089-44e1-994f-75d2b4570fce\") " pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:18:36.191092 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.190972 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8af4784e-62fd-4a92-ba60-c91f69a2322b-hosts-file\") pod \"node-resolver-78f24\" (UID: \"8af4784e-62fd-4a92-ba60-c91f69a2322b\") " pod="openshift-dns/node-resolver-78f24" Jul 05 13:18:36.191092 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.190988 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-etc-kubernetes\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.191092 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.191083 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd1a8674-00ab-40d3-b169-f5857832053a-etc-kubernetes\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.191348 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.191322 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8af4784e-62fd-4a92-ba60-c91f69a2322b-tmp-dir\") pod \"node-resolver-78f24\" (UID: \"8af4784e-62fd-4a92-ba60-c91f69a2322b\") " pod="openshift-dns/node-resolver-78f24" Jul 05 13:18:36.191416 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.191369 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d164f387-3a93-465a-a587-ac48c676e2f6-system-cni-dir\") pod \"multus-additional-cni-plugins-wk7fc\" (UID: \"d164f387-3a93-465a-a587-ac48c676e2f6\") " pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.191472 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.191420 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/32a4a45d-3b64-476e-9147-09693f809841-kubelet-config\") pod \"global-pull-secret-syncer-k92vt\" (UID: \"32a4a45d-3b64-476e-9147-09693f809841\") " pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:18:36.191472 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.191463 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/32a4a45d-3b64-476e-9147-09693f809841-kubelet-config\") pod \"global-pull-secret-syncer-k92vt\" (UID: \"32a4a45d-3b64-476e-9147-09693f809841\") " pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:18:36.191580 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.191459 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/32a4a45d-3b64-476e-9147-09693f809841-dbus\") pod \"global-pull-secret-syncer-k92vt\" (UID: \"32a4a45d-3b64-476e-9147-09693f809841\") " pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:18:36.191580 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.191429 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d164f387-3a93-465a-a587-ac48c676e2f6-system-cni-dir\") pod \"multus-additional-cni-plugins-wk7fc\" (UID: \"d164f387-3a93-465a-a587-ac48c676e2f6\") " pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.191580 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.191509 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cd1a8674-00ab-40d3-b169-f5857832053a-etc-tuned\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.191580 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.191517 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/df0a5b01-f000-4879-8fcc-175f435bce3e-agent-certs\") pod \"konnectivity-agent-qmf78\" (UID: \"df0a5b01-f000-4879-8fcc-175f435bce3e\") " pod="kube-system/konnectivity-agent-qmf78" Jul 05 13:18:36.191580 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.191541 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd1a8674-00ab-40d3-b169-f5857832053a-tmp\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.191580 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.191563 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7364f530-c4ff-47ba-a86b-904542a26d07-iptables-alerter-script\") pod \"iptables-alerter-xj828\" (UID: \"7364f530-c4ff-47ba-a86b-904542a26d07\") " pod="openshift-network-operator/iptables-alerter-xj828" Jul 05 13:18:36.191899 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.191589 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-host-cni-netd\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.191899 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.191622 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/32a4a45d-3b64-476e-9147-09693f809841-dbus\") pod \"global-pull-secret-syncer-k92vt\" (UID: \"32a4a45d-3b64-476e-9147-09693f809841\") " pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:18:36.191899 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.191659 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-host-cni-netd\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.191899 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.191664 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.191899 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.191620 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.191899 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.191703 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fed1aa39-8ecb-4770-9dea-c73e7b76386a-etc-selinux\") pod \"aws-ebs-csi-driver-node-gkcrw\" (UID: \"fed1aa39-8ecb-4770-9dea-c73e7b76386a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" Jul 05 13:18:36.191899 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.191732 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvcfs\" (UniqueName: \"kubernetes.io/projected/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-kube-api-access-qvcfs\") pod \"network-metrics-daemon-9sgqr\" (UID: \"bcbf773f-ee0a-42bd-8983-a1a3bb066e5d\") " pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:18:36.191899 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.191755 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-host-slash\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.191899 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.191780 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-etc-openvswitch\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.191899 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.191829 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-run-openvswitch\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.191899 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.191864 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-os-release\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.192666 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.191970 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-run-openvswitch\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.192666 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.191953 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-etc-openvswitch\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.192666 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.192005 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a447ce81-b3cb-4269-8d46-e6e612b89e0b-host-slash\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.192666 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.192075 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fed1aa39-8ecb-4770-9dea-c73e7b76386a-etc-selinux\") pod \"aws-ebs-csi-driver-node-gkcrw\" (UID: \"fed1aa39-8ecb-4770-9dea-c73e7b76386a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" Jul 05 13:18:36.192666 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.192336 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a447ce81-b3cb-4269-8d46-e6e612b89e0b-ovn-node-metrics-cert\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.192966 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.192927 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7364f530-c4ff-47ba-a86b-904542a26d07-iptables-alerter-script\") pod \"iptables-alerter-xj828\" (UID: \"7364f530-c4ff-47ba-a86b-904542a26d07\") " pod="openshift-network-operator/iptables-alerter-xj828" Jul 05 13:18:36.194183 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.194159 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd1a8674-00ab-40d3-b169-f5857832053a-tmp\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.194298 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.194259 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cd1a8674-00ab-40d3-b169-f5857832053a-etc-tuned\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.195504 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.195477 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snv5l\" (UniqueName: \"kubernetes.io/projected/d164f387-3a93-465a-a587-ac48c676e2f6-kube-api-access-snv5l\") pod \"multus-additional-cni-plugins-wk7fc\" (UID: \"d164f387-3a93-465a-a587-ac48c676e2f6\") " pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.196960 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.196934 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbwhx\" (UniqueName: \"kubernetes.io/projected/7364f530-c4ff-47ba-a86b-904542a26d07-kube-api-access-wbwhx\") pod \"iptables-alerter-xj828\" (UID: \"7364f530-c4ff-47ba-a86b-904542a26d07\") " pod="openshift-network-operator/iptables-alerter-xj828" Jul 05 13:18:36.207723 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.207689 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7pc4\" (UniqueName: \"kubernetes.io/projected/8af4784e-62fd-4a92-ba60-c91f69a2322b-kube-api-access-s7pc4\") pod \"node-resolver-78f24\" (UID: \"8af4784e-62fd-4a92-ba60-c91f69a2322b\") " pod="openshift-dns/node-resolver-78f24" Jul 05 13:18:36.211686 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:36.211654 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:18:36.211686 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:36.211689 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:18:36.211910 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:36.211705 2567 projected.go:196] Error preparing data for projected volume kube-api-access-x6w76 for pod openshift-network-diagnostics/network-check-target-j5rvd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:36.211910 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:36.211780 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d509ded-4089-44e1-994f-75d2b4570fce-kube-api-access-x6w76 podName:2d509ded-4089-44e1-994f-75d2b4570fce nodeName:}" failed. No retries permitted until 2026-07-05 13:18:36.711758737 +0000 UTC m=+3.052776513 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-x6w76" (UniqueName: "kubernetes.io/projected/2d509ded-4089-44e1-994f-75d2b4570fce-kube-api-access-x6w76") pod "network-check-target-j5rvd" (UID: "2d509ded-4089-44e1-994f-75d2b4570fce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:36.212038 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.212017 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tngz8\" (UniqueName: \"kubernetes.io/projected/fed1aa39-8ecb-4770-9dea-c73e7b76386a-kube-api-access-tngz8\") pod \"aws-ebs-csi-driver-node-gkcrw\" (UID: \"fed1aa39-8ecb-4770-9dea-c73e7b76386a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" Jul 05 13:18:36.212479 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.212261 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrttb\" (UniqueName: \"kubernetes.io/projected/cd1a8674-00ab-40d3-b169-f5857832053a-kube-api-access-nrttb\") pod \"tuned-mhlfp\" (UID: \"cd1a8674-00ab-40d3-b169-f5857832053a\") " pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.212769 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.212718 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpbxg\" (UniqueName: \"kubernetes.io/projected/84b36d5b-eb96-4252-a0d2-a1f5d8c58a4e-kube-api-access-xpbxg\") pod \"node-ca-8s4z9\" (UID: \"84b36d5b-eb96-4252-a0d2-a1f5d8c58a4e\") " pod="openshift-image-registry/node-ca-8s4z9" Jul 05 13:18:36.214677 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.214616 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-119.ec2.internal" event={"ID":"7a0d072d628a3c33122d4e7cac8d89f1","Type":"ContainerStarted","Data":"a073fe1ee43ec1ed613b4d8db25671a4d3e1b37399718653b54a54d097473f6f"} Jul 05 13:18:36.214782 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.214674 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvcfs\" (UniqueName: \"kubernetes.io/projected/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-kube-api-access-qvcfs\") pod \"network-metrics-daemon-9sgqr\" (UID: \"bcbf773f-ee0a-42bd-8983-a1a3bb066e5d\") " pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:18:36.215297 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.215265 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d458z\" (UniqueName: \"kubernetes.io/projected/a447ce81-b3cb-4269-8d46-e6e612b89e0b-kube-api-access-d458z\") pod \"ovnkube-node-swckn\" (UID: \"a447ce81-b3cb-4269-8d46-e6e612b89e0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.215928 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.215896 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-119.ec2.internal" event={"ID":"a1a32c2f892e1c59cdaebdb361cae096","Type":"ContainerStarted","Data":"2b86c4900594d5020651d115e941605ef0448f70d1c69545a28e0e7b9288c7b0"} Jul 05 13:18:36.292294 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.292250 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f2d4dd11-346b-496e-886b-9238eaa91508-multus-daemon-config\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.292493 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.292306 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g688n\" (UniqueName: \"kubernetes.io/projected/f2d4dd11-346b-496e-886b-9238eaa91508-kube-api-access-g688n\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.292493 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.292337 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-host-var-lib-kubelet\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.292493 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.292359 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-hostroot\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.292493 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.292385 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-host-run-k8s-cni-cncf-io\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.292493 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.292426 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-host-var-lib-cni-bin\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.292493 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.292454 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-etc-kubernetes\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.292493 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.292484 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-system-cni-dir\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.292879 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.292504 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2d4dd11-346b-496e-886b-9238eaa91508-cni-binary-copy\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.292879 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.292537 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-multus-conf-dir\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.292879 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.292579 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-os-release\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.292879 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.292595 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-multus-socket-dir-parent\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.292879 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.292622 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-host-run-netns\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.292879 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.292652 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-host-var-lib-cni-multus\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.292879 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.292682 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-cnibin\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.292879 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.292707 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-host-run-multus-certs\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.292879 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.292733 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-multus-cni-dir\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.292879 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.292853 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-multus-cni-dir\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.293513 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.293020 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f2d4dd11-346b-496e-886b-9238eaa91508-multus-daemon-config\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.293513 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.293204 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-multus-conf-dir\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.293513 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.293263 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-os-release\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.293513 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.293310 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-multus-socket-dir-parent\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.293513 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.293337 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-host-run-netns\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.293513 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.293364 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-host-var-lib-cni-multus\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.293513 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.293411 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-cnibin\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.293513 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.293473 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-host-run-multus-certs\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.293759 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.293517 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2d4dd11-346b-496e-886b-9238eaa91508-cni-binary-copy\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.293759 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.293534 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-host-run-k8s-cni-cncf-io\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.293759 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.293561 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-hostroot\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.293759 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.293585 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-host-var-lib-cni-bin\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.293759 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.293638 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-system-cni-dir\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.293759 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.293675 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-etc-kubernetes\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.293759 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.293708 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2d4dd11-346b-496e-886b-9238eaa91508-host-var-lib-kubelet\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.301925 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.301886 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g688n\" (UniqueName: \"kubernetes.io/projected/f2d4dd11-346b-496e-886b-9238eaa91508-kube-api-access-g688n\") pod \"multus-k882t\" (UID: \"f2d4dd11-346b-496e-886b-9238eaa91508\") " pod="openshift-multus/multus-k882t" Jul 05 13:18:36.378565 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.378527 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" Jul 05 13:18:36.390047 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.389956 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qmf78" Jul 05 13:18:36.399121 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.399089 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xj828" Jul 05 13:18:36.406239 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.406196 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:18:36.413054 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.413028 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-78f24" Jul 05 13:18:36.419713 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.419685 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8s4z9" Jul 05 13:18:36.432423 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.432395 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" Jul 05 13:18:36.441180 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.441146 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wk7fc" Jul 05 13:18:36.447329 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.447305 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k882t" Jul 05 13:18:36.695665 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.695624 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-metrics-certs\") pod \"network-metrics-daemon-9sgqr\" (UID: \"bcbf773f-ee0a-42bd-8983-a1a3bb066e5d\") " pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:18:36.695852 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.695682 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32a4a45d-3b64-476e-9147-09693f809841-original-pull-secret\") pod \"global-pull-secret-syncer-k92vt\" (UID: \"32a4a45d-3b64-476e-9147-09693f809841\") " pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:18:36.695852 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:36.695806 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:36.695852 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:36.695829 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:36.695992 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:36.695897 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32a4a45d-3b64-476e-9147-09693f809841-original-pull-secret podName:32a4a45d-3b64-476e-9147-09693f809841 nodeName:}" failed. No retries permitted until 2026-07-05 13:18:37.695874 +0000 UTC m=+4.036891755 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/32a4a45d-3b64-476e-9147-09693f809841-original-pull-secret") pod "global-pull-secret-syncer-k92vt" (UID: "32a4a45d-3b64-476e-9147-09693f809841") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:36.695992 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:36.695919 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-metrics-certs podName:bcbf773f-ee0a-42bd-8983-a1a3bb066e5d nodeName:}" failed. No retries permitted until 2026-07-05 13:18:37.695908169 +0000 UTC m=+4.036925920 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-metrics-certs") pod "network-metrics-daemon-9sgqr" (UID: "bcbf773f-ee0a-42bd-8983-a1a3bb066e5d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:36.796438 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:36.796398 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6w76\" (UniqueName: \"kubernetes.io/projected/2d509ded-4089-44e1-994f-75d2b4570fce-kube-api-access-x6w76\") pod \"network-check-target-j5rvd\" (UID: \"2d509ded-4089-44e1-994f-75d2b4570fce\") " pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:18:36.796619 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:36.796543 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:18:36.796619 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:36.796565 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:18:36.796619 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:36.796575 2567 projected.go:196] Error preparing data for projected volume kube-api-access-x6w76 for pod openshift-network-diagnostics/network-check-target-j5rvd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:36.796736 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:36.796632 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d509ded-4089-44e1-994f-75d2b4570fce-kube-api-access-x6w76 podName:2d509ded-4089-44e1-994f-75d2b4570fce nodeName:}" failed. No retries permitted until 2026-07-05 13:18:37.796618704 +0000 UTC m=+4.137636454 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-x6w76" (UniqueName: "kubernetes.io/projected/2d509ded-4089-44e1-994f-75d2b4570fce-kube-api-access-x6w76") pod "network-check-target-j5rvd" (UID: "2d509ded-4089-44e1-994f-75d2b4570fce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:37.095043 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:37.094837 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd164f387_3a93_465a_a587_ac48c676e2f6.slice/crio-3799c635cf9f85cd73facf4474ada280067f742a617070f4f1f9c5dcbf73630e WatchSource:0}: Error finding container 3799c635cf9f85cd73facf4474ada280067f742a617070f4f1f9c5dcbf73630e: Status 404 returned error can't find the container with id 3799c635cf9f85cd73facf4474ada280067f742a617070f4f1f9c5dcbf73630e Jul 05 13:18:37.098012 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:18:37.097987 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8af4784e_62fd_4a92_ba60_c91f69a2322b.slice/crio-8340e157385cd1bdd264a5cea0ca1b497e383cce99ec661978c2f273cb207e21 WatchSource:0}: Error finding container 8340e157385cd1bdd264a5cea0ca1b497e383cce99ec661978c2f273cb207e21: Status 404 returned error can't find the container with id 8340e157385cd1bdd264a5cea0ca1b497e383cce99ec661978c2f273cb207e21 Jul 05 13:18:37.133695 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:37.133628 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-04 13:13:36 +0000 UTC" deadline="2028-04-12 00:02:31.97688842 +0000 UTC" Jul 05 13:18:37.133695 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:37.133694 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15514h43m54.843199073s" Jul 05 13:18:37.209994 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:37.209965 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:18:37.210164 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:37.210104 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sgqr" podUID="bcbf773f-ee0a-42bd-8983-a1a3bb066e5d" Jul 05 13:18:37.219567 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:37.219531 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8s4z9" event={"ID":"84b36d5b-eb96-4252-a0d2-a1f5d8c58a4e","Type":"ContainerStarted","Data":"36db4a7e0c260c5b83fa5afe0efb396e11e61d11aa641f0100f2a284c9e89335"} Jul 05 13:18:37.220804 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:37.220772 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" event={"ID":"fed1aa39-8ecb-4770-9dea-c73e7b76386a","Type":"ContainerStarted","Data":"9365d83bdcf06cda1fc5919227bbb09f64f98b27846e3bf977e8772a6a8a1a2b"} Jul 05 13:18:37.221934 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:37.221906 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k882t" event={"ID":"f2d4dd11-346b-496e-886b-9238eaa91508","Type":"ContainerStarted","Data":"5bdada1ef6e45110f77b250ca072b4385056b18d5b8f456df8da09d3be21d13e"} Jul 05 13:18:37.223021 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:37.222998 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk7fc" event={"ID":"d164f387-3a93-465a-a587-ac48c676e2f6","Type":"ContainerStarted","Data":"3799c635cf9f85cd73facf4474ada280067f742a617070f4f1f9c5dcbf73630e"} Jul 05 13:18:37.224161 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:37.224130 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xj828" event={"ID":"7364f530-c4ff-47ba-a86b-904542a26d07","Type":"ContainerStarted","Data":"353c7f9e904e55a8a87a646464e8a489eb12aeef0e5768fc5b2042f9604bebb0"} Jul 05 13:18:37.225326 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:37.225303 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qmf78" event={"ID":"df0a5b01-f000-4879-8fcc-175f435bce3e","Type":"ContainerStarted","Data":"1a12c834d8cd021d3ee7561dcb47cc84648a1dc8a6c6590ba093dcdb3e2204ed"} Jul 05 13:18:37.226454 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:37.226433 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" event={"ID":"cd1a8674-00ab-40d3-b169-f5857832053a","Type":"ContainerStarted","Data":"4d597a5d11479771fe9e1d04d520e491307392f4968b6c3488423a02595d1fe7"} Jul 05 13:18:37.227574 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:37.227550 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swckn" event={"ID":"a447ce81-b3cb-4269-8d46-e6e612b89e0b","Type":"ContainerStarted","Data":"cc627dc0298303ca9b873f7b567f1336614421ba3fd44fc4c42a97e726f3988e"} Jul 05 13:18:37.228710 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:37.228684 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-78f24" event={"ID":"8af4784e-62fd-4a92-ba60-c91f69a2322b","Type":"ContainerStarted","Data":"8340e157385cd1bdd264a5cea0ca1b497e383cce99ec661978c2f273cb207e21"} Jul 05 13:18:37.230476 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:37.230450 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-119.ec2.internal" event={"ID":"a1a32c2f892e1c59cdaebdb361cae096","Type":"ContainerStarted","Data":"e520a9886d48ec67d4a91c1d9b13d90a73ba6fecdd99f6fc42ebd75aec1f29ad"} Jul 05 13:18:37.703531 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:37.703490 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-metrics-certs\") pod \"network-metrics-daemon-9sgqr\" (UID: \"bcbf773f-ee0a-42bd-8983-a1a3bb066e5d\") " pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:18:37.703706 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:37.703565 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32a4a45d-3b64-476e-9147-09693f809841-original-pull-secret\") pod \"global-pull-secret-syncer-k92vt\" (UID: \"32a4a45d-3b64-476e-9147-09693f809841\") " pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:18:37.703706 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:37.703686 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:37.703877 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:37.703768 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32a4a45d-3b64-476e-9147-09693f809841-original-pull-secret podName:32a4a45d-3b64-476e-9147-09693f809841 nodeName:}" failed. No retries permitted until 2026-07-05 13:18:39.703731385 +0000 UTC m=+6.044749152 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/32a4a45d-3b64-476e-9147-09693f809841-original-pull-secret") pod "global-pull-secret-syncer-k92vt" (UID: "32a4a45d-3b64-476e-9147-09693f809841") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:37.704221 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:37.704192 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:37.704308 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:37.704263 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-metrics-certs podName:bcbf773f-ee0a-42bd-8983-a1a3bb066e5d nodeName:}" failed. No retries permitted until 2026-07-05 13:18:39.704245064 +0000 UTC m=+6.045262820 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-metrics-certs") pod "network-metrics-daemon-9sgqr" (UID: "bcbf773f-ee0a-42bd-8983-a1a3bb066e5d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:37.805034 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:37.804996 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6w76\" (UniqueName: \"kubernetes.io/projected/2d509ded-4089-44e1-994f-75d2b4570fce-kube-api-access-x6w76\") pod \"network-check-target-j5rvd\" (UID: \"2d509ded-4089-44e1-994f-75d2b4570fce\") " pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:18:37.805213 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:37.805194 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:18:37.805213 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:37.805213 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:18:37.805320 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:37.805225 2567 projected.go:196] Error preparing data for projected volume kube-api-access-x6w76 for pod openshift-network-diagnostics/network-check-target-j5rvd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:37.805320 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:37.805312 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d509ded-4089-44e1-994f-75d2b4570fce-kube-api-access-x6w76 podName:2d509ded-4089-44e1-994f-75d2b4570fce nodeName:}" failed. No retries permitted until 2026-07-05 13:18:39.805293452 +0000 UTC m=+6.146311216 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-x6w76" (UniqueName: "kubernetes.io/projected/2d509ded-4089-44e1-994f-75d2b4570fce-kube-api-access-x6w76") pod "network-check-target-j5rvd" (UID: "2d509ded-4089-44e1-994f-75d2b4570fce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:38.134565 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:38.134454 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-04 13:13:36 +0000 UTC" deadline="2028-01-26 15:20:55.093327925 +0000 UTC" Jul 05 13:18:38.134565 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:38.134495 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13682h2m16.958837558s" Jul 05 13:18:38.209669 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:38.209636 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:18:38.209847 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:38.209785 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k92vt" podUID="32a4a45d-3b64-476e-9147-09693f809841" Jul 05 13:18:38.210354 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:38.210332 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:18:38.210468 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:38.210447 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j5rvd" podUID="2d509ded-4089-44e1-994f-75d2b4570fce" Jul 05 13:18:38.259266 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:38.257892 2567 generic.go:356] "Generic (PLEG): container finished" podID="7a0d072d628a3c33122d4e7cac8d89f1" containerID="ce832cb1be0ec43e3ebb8c70793eca72c776d9970d4b6257a81d3b3a08cc56e9" exitCode=0 Jul 05 13:18:38.259266 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:38.258998 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-119.ec2.internal" event={"ID":"7a0d072d628a3c33122d4e7cac8d89f1","Type":"ContainerDied","Data":"ce832cb1be0ec43e3ebb8c70793eca72c776d9970d4b6257a81d3b3a08cc56e9"} Jul 05 13:18:38.276127 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:38.276057 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-119.ec2.internal" podStartSLOduration=3.276034365 podStartE2EDuration="3.276034365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:18:37.246245432 +0000 UTC m=+3.587263203" watchObservedRunningTime="2026-07-05 13:18:38.276034365 +0000 UTC m=+4.617052139" Jul 05 13:18:39.209348 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:39.209279 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:18:39.209902 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:39.209438 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sgqr" podUID="bcbf773f-ee0a-42bd-8983-a1a3bb066e5d" Jul 05 13:18:39.270545 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:39.270444 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-119.ec2.internal" event={"ID":"7a0d072d628a3c33122d4e7cac8d89f1","Type":"ContainerStarted","Data":"6eb74bf694e9544f15f5627193af861a07912ddae55b422dc3192a775c5c1b13"} Jul 05 13:18:39.288579 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:39.287530 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-119.ec2.internal" podStartSLOduration=4.287509965 podStartE2EDuration="4.287509965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:18:39.2870625 +0000 UTC m=+5.628080274" watchObservedRunningTime="2026-07-05 13:18:39.287509965 +0000 UTC m=+5.628527737" Jul 05 13:18:39.724016 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:39.723973 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-metrics-certs\") pod \"network-metrics-daemon-9sgqr\" (UID: \"bcbf773f-ee0a-42bd-8983-a1a3bb066e5d\") " pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:18:39.724219 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:39.724036 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32a4a45d-3b64-476e-9147-09693f809841-original-pull-secret\") pod \"global-pull-secret-syncer-k92vt\" (UID: \"32a4a45d-3b64-476e-9147-09693f809841\") " pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:18:39.724219 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:39.724161 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:39.724337 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:39.724231 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32a4a45d-3b64-476e-9147-09693f809841-original-pull-secret podName:32a4a45d-3b64-476e-9147-09693f809841 nodeName:}" failed. No retries permitted until 2026-07-05 13:18:43.724213174 +0000 UTC m=+10.065230927 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/32a4a45d-3b64-476e-9147-09693f809841-original-pull-secret") pod "global-pull-secret-syncer-k92vt" (UID: "32a4a45d-3b64-476e-9147-09693f809841") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:39.724646 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:39.724626 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:39.724877 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:39.724683 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-metrics-certs podName:bcbf773f-ee0a-42bd-8983-a1a3bb066e5d nodeName:}" failed. No retries permitted until 2026-07-05 13:18:43.724667181 +0000 UTC m=+10.065684954 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-metrics-certs") pod "network-metrics-daemon-9sgqr" (UID: "bcbf773f-ee0a-42bd-8983-a1a3bb066e5d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:39.824616 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:39.824572 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6w76\" (UniqueName: \"kubernetes.io/projected/2d509ded-4089-44e1-994f-75d2b4570fce-kube-api-access-x6w76\") pod \"network-check-target-j5rvd\" (UID: \"2d509ded-4089-44e1-994f-75d2b4570fce\") " pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:18:39.824840 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:39.824820 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:18:39.824903 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:39.824847 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:18:39.824903 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:39.824860 2567 projected.go:196] Error preparing data for projected volume kube-api-access-x6w76 for pod openshift-network-diagnostics/network-check-target-j5rvd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:39.824996 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:39.824918 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d509ded-4089-44e1-994f-75d2b4570fce-kube-api-access-x6w76 podName:2d509ded-4089-44e1-994f-75d2b4570fce nodeName:}" failed. No retries permitted until 2026-07-05 13:18:43.824900351 +0000 UTC m=+10.165918107 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-x6w76" (UniqueName: "kubernetes.io/projected/2d509ded-4089-44e1-994f-75d2b4570fce-kube-api-access-x6w76") pod "network-check-target-j5rvd" (UID: "2d509ded-4089-44e1-994f-75d2b4570fce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:40.210907 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:40.210873 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:18:40.211306 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:40.211010 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j5rvd" podUID="2d509ded-4089-44e1-994f-75d2b4570fce" Jul 05 13:18:40.211306 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:40.211060 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:18:40.211306 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:40.211131 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k92vt" podUID="32a4a45d-3b64-476e-9147-09693f809841" Jul 05 13:18:41.209508 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:41.209470 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:18:41.209708 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:41.209648 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sgqr" podUID="bcbf773f-ee0a-42bd-8983-a1a3bb066e5d" Jul 05 13:18:42.210896 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:42.210853 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:18:42.211351 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:42.210865 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:18:42.211351 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:42.210992 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k92vt" podUID="32a4a45d-3b64-476e-9147-09693f809841" Jul 05 13:18:42.211351 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:42.211050 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j5rvd" podUID="2d509ded-4089-44e1-994f-75d2b4570fce" Jul 05 13:18:43.209974 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:43.209939 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:18:43.210208 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:43.210109 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sgqr" podUID="bcbf773f-ee0a-42bd-8983-a1a3bb066e5d" Jul 05 13:18:43.758299 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:43.758266 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-metrics-certs\") pod \"network-metrics-daemon-9sgqr\" (UID: \"bcbf773f-ee0a-42bd-8983-a1a3bb066e5d\") " pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:18:43.758801 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:43.758318 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32a4a45d-3b64-476e-9147-09693f809841-original-pull-secret\") pod \"global-pull-secret-syncer-k92vt\" (UID: \"32a4a45d-3b64-476e-9147-09693f809841\") " pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:18:43.758801 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:43.758431 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:43.758801 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:43.758484 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32a4a45d-3b64-476e-9147-09693f809841-original-pull-secret podName:32a4a45d-3b64-476e-9147-09693f809841 nodeName:}" failed. No retries permitted until 2026-07-05 13:18:51.758466316 +0000 UTC m=+18.099484067 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/32a4a45d-3b64-476e-9147-09693f809841-original-pull-secret") pod "global-pull-secret-syncer-k92vt" (UID: "32a4a45d-3b64-476e-9147-09693f809841") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:43.758801 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:43.758533 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:43.758801 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:43.758557 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-metrics-certs podName:bcbf773f-ee0a-42bd-8983-a1a3bb066e5d nodeName:}" failed. No retries permitted until 2026-07-05 13:18:51.758546556 +0000 UTC m=+18.099564307 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-metrics-certs") pod "network-metrics-daemon-9sgqr" (UID: "bcbf773f-ee0a-42bd-8983-a1a3bb066e5d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:43.858794 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:43.858754 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6w76\" (UniqueName: \"kubernetes.io/projected/2d509ded-4089-44e1-994f-75d2b4570fce-kube-api-access-x6w76\") pod \"network-check-target-j5rvd\" (UID: \"2d509ded-4089-44e1-994f-75d2b4570fce\") " pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:18:43.858959 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:43.858924 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:18:43.858959 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:43.858939 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:18:43.858959 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:43.858949 2567 projected.go:196] Error preparing data for projected volume kube-api-access-x6w76 for pod openshift-network-diagnostics/network-check-target-j5rvd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:43.859054 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:43.858991 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d509ded-4089-44e1-994f-75d2b4570fce-kube-api-access-x6w76 podName:2d509ded-4089-44e1-994f-75d2b4570fce nodeName:}" failed. No retries permitted until 2026-07-05 13:18:51.858979074 +0000 UTC m=+18.199996824 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-x6w76" (UniqueName: "kubernetes.io/projected/2d509ded-4089-44e1-994f-75d2b4570fce-kube-api-access-x6w76") pod "network-check-target-j5rvd" (UID: "2d509ded-4089-44e1-994f-75d2b4570fce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:44.210476 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:44.210443 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:18:44.210643 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:44.210494 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:18:44.210643 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:44.210566 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j5rvd" podUID="2d509ded-4089-44e1-994f-75d2b4570fce" Jul 05 13:18:44.210725 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:44.210667 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k92vt" podUID="32a4a45d-3b64-476e-9147-09693f809841" Jul 05 13:18:45.209982 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:45.209950 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:18:45.210443 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:45.210087 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sgqr" podUID="bcbf773f-ee0a-42bd-8983-a1a3bb066e5d" Jul 05 13:18:46.212436 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:46.209842 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:18:46.212436 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:46.209901 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:18:46.212436 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:46.210011 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j5rvd" podUID="2d509ded-4089-44e1-994f-75d2b4570fce" Jul 05 13:18:46.212436 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:46.212370 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k92vt" podUID="32a4a45d-3b64-476e-9147-09693f809841" Jul 05 13:18:47.209482 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:47.209448 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:18:47.209665 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:47.209585 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sgqr" podUID="bcbf773f-ee0a-42bd-8983-a1a3bb066e5d" Jul 05 13:18:48.210023 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:48.209980 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:18:48.210481 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:48.209989 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:18:48.210481 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:48.210108 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k92vt" podUID="32a4a45d-3b64-476e-9147-09693f809841" Jul 05 13:18:48.210481 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:48.210212 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j5rvd" podUID="2d509ded-4089-44e1-994f-75d2b4570fce" Jul 05 13:18:49.209130 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:49.209094 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:18:49.209311 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:49.209223 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sgqr" podUID="bcbf773f-ee0a-42bd-8983-a1a3bb066e5d" Jul 05 13:18:50.209090 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:50.209053 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:18:50.209521 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:50.209171 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k92vt" podUID="32a4a45d-3b64-476e-9147-09693f809841" Jul 05 13:18:50.209521 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:50.209297 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:18:50.209521 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:50.209416 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j5rvd" podUID="2d509ded-4089-44e1-994f-75d2b4570fce" Jul 05 13:18:51.210051 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:51.210013 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:18:51.210560 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:51.210129 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sgqr" podUID="bcbf773f-ee0a-42bd-8983-a1a3bb066e5d" Jul 05 13:18:51.821923 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:51.821887 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32a4a45d-3b64-476e-9147-09693f809841-original-pull-secret\") pod \"global-pull-secret-syncer-k92vt\" (UID: \"32a4a45d-3b64-476e-9147-09693f809841\") " pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:18:51.822166 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:51.821962 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-metrics-certs\") pod \"network-metrics-daemon-9sgqr\" (UID: \"bcbf773f-ee0a-42bd-8983-a1a3bb066e5d\") " pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:18:51.822166 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:51.822043 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:51.822166 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:51.822117 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32a4a45d-3b64-476e-9147-09693f809841-original-pull-secret podName:32a4a45d-3b64-476e-9147-09693f809841 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:07.822082354 +0000 UTC m=+34.163100104 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/32a4a45d-3b64-476e-9147-09693f809841-original-pull-secret") pod "global-pull-secret-syncer-k92vt" (UID: "32a4a45d-3b64-476e-9147-09693f809841") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:51.822166 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:51.822043 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:51.822166 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:51.822165 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-metrics-certs podName:bcbf773f-ee0a-42bd-8983-a1a3bb066e5d nodeName:}" failed. No retries permitted until 2026-07-05 13:19:07.822151781 +0000 UTC m=+34.163169534 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-metrics-certs") pod "network-metrics-daemon-9sgqr" (UID: "bcbf773f-ee0a-42bd-8983-a1a3bb066e5d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:51.922319 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:51.922297 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6w76\" (UniqueName: \"kubernetes.io/projected/2d509ded-4089-44e1-994f-75d2b4570fce-kube-api-access-x6w76\") pod \"network-check-target-j5rvd\" (UID: \"2d509ded-4089-44e1-994f-75d2b4570fce\") " pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:18:51.922471 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:51.922453 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:18:51.922510 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:51.922475 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:18:51.922510 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:51.922484 2567 projected.go:196] Error preparing data for projected volume kube-api-access-x6w76 for pod openshift-network-diagnostics/network-check-target-j5rvd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:51.922581 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:51.922531 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d509ded-4089-44e1-994f-75d2b4570fce-kube-api-access-x6w76 podName:2d509ded-4089-44e1-994f-75d2b4570fce nodeName:}" failed. No retries permitted until 2026-07-05 13:19:07.9225183 +0000 UTC m=+34.263536050 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-x6w76" (UniqueName: "kubernetes.io/projected/2d509ded-4089-44e1-994f-75d2b4570fce-kube-api-access-x6w76") pod "network-check-target-j5rvd" (UID: "2d509ded-4089-44e1-994f-75d2b4570fce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:52.209985 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:52.209950 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:18:52.210174 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:52.210076 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k92vt" podUID="32a4a45d-3b64-476e-9147-09693f809841" Jul 05 13:18:52.210174 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:52.210136 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:18:52.210575 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:52.210266 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j5rvd" podUID="2d509ded-4089-44e1-994f-75d2b4570fce" Jul 05 13:18:53.209944 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:53.209906 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:18:53.210140 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:53.210036 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sgqr" podUID="bcbf773f-ee0a-42bd-8983-a1a3bb066e5d" Jul 05 13:18:54.210798 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:54.210757 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:18:54.211271 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:54.210809 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:18:54.211271 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:54.210889 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k92vt" podUID="32a4a45d-3b64-476e-9147-09693f809841" Jul 05 13:18:54.211271 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:54.211010 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j5rvd" podUID="2d509ded-4089-44e1-994f-75d2b4570fce" Jul 05 13:18:55.209558 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:55.209521 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:18:55.209756 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:55.209676 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sgqr" podUID="bcbf773f-ee0a-42bd-8983-a1a3bb066e5d" Jul 05 13:18:56.209559 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:56.209523 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:18:56.210152 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:56.209657 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j5rvd" podUID="2d509ded-4089-44e1-994f-75d2b4570fce" Jul 05 13:18:56.210152 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:56.209704 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:18:56.210152 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:56.209878 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k92vt" podUID="32a4a45d-3b64-476e-9147-09693f809841" Jul 05 13:18:57.209820 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:57.209459 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:18:57.210502 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:57.209927 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sgqr" podUID="bcbf773f-ee0a-42bd-8983-a1a3bb066e5d" Jul 05 13:18:57.305188 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:57.305090 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qmf78" event={"ID":"df0a5b01-f000-4879-8fcc-175f435bce3e","Type":"ContainerStarted","Data":"8b6508fe4f65cf542d34a5a9c32b82971bd5465613cf3a322ba5eafed3f2d528"} Jul 05 13:18:57.306807 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:57.306778 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" event={"ID":"cd1a8674-00ab-40d3-b169-f5857832053a","Type":"ContainerStarted","Data":"349e83a7a7de290c6a298230b22628937763ebe6766c890adb8aa4773de2ac86"} Jul 05 13:18:57.310136 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:57.310113 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swckn_a447ce81-b3cb-4269-8d46-e6e612b89e0b/ovn-acl-logging/0.log" Jul 05 13:18:57.310505 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:57.310478 2567 generic.go:356] "Generic (PLEG): container finished" podID="a447ce81-b3cb-4269-8d46-e6e612b89e0b" containerID="57243dc0d2ebe29ec8c1c7466017cd45827efb4b28f2c089844b5a985c941eee" exitCode=1 Jul 05 13:18:57.310654 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:57.310552 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swckn" event={"ID":"a447ce81-b3cb-4269-8d46-e6e612b89e0b","Type":"ContainerStarted","Data":"15d26b90cc401c6b720d1edf68a15a1b6fd9af3ccc68e2862ae3975482167616"} Jul 05 13:18:57.310654 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:57.310627 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swckn" event={"ID":"a447ce81-b3cb-4269-8d46-e6e612b89e0b","Type":"ContainerStarted","Data":"48491aee6d1e6e5133da7241526c48df92bd1efee8781be08e418ad0b209f1c8"} Jul 05 13:18:57.310785 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:57.310670 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swckn" event={"ID":"a447ce81-b3cb-4269-8d46-e6e612b89e0b","Type":"ContainerStarted","Data":"531a16bdeb0c05cf8489994a8c6d28c4ffbc61daf8e5ecfa4115f583d59d957f"} Jul 05 13:18:57.310785 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:57.310716 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swckn" event={"ID":"a447ce81-b3cb-4269-8d46-e6e612b89e0b","Type":"ContainerStarted","Data":"fcbd08c76526d2169264d0a88e8bf207e76b51f55a9be26d1539dc3c4aa03325"} Jul 05 13:18:57.310785 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:57.310772 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swckn" event={"ID":"a447ce81-b3cb-4269-8d46-e6e612b89e0b","Type":"ContainerDied","Data":"57243dc0d2ebe29ec8c1c7466017cd45827efb4b28f2c089844b5a985c941eee"} Jul 05 13:18:57.310941 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:57.310816 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swckn" event={"ID":"a447ce81-b3cb-4269-8d46-e6e612b89e0b","Type":"ContainerStarted","Data":"dd588ed757c85f590643d1340a315ffbfd843124cffb691b6b241aac760be3bf"} Jul 05 13:18:57.312130 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:57.312099 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-78f24" event={"ID":"8af4784e-62fd-4a92-ba60-c91f69a2322b","Type":"ContainerStarted","Data":"c776784d33fd998dfe1369a11b4b552d92c77d5d9efe29a9c0ebf645c6473be5"} Jul 05 13:18:57.313812 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:57.313781 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8s4z9" event={"ID":"84b36d5b-eb96-4252-a0d2-a1f5d8c58a4e","Type":"ContainerStarted","Data":"0eb36c18404eff8907dcac73583213e3274bc602895acba654db7f2c0a0e0a94"} Jul 05 13:18:57.315437 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:57.315396 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" event={"ID":"fed1aa39-8ecb-4770-9dea-c73e7b76386a","Type":"ContainerStarted","Data":"c47f7442bd5a7bd8ed16f418aa56f171ffd0f128f20316a85b48da81ebe830dd"} Jul 05 13:18:57.317245 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:57.317211 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k882t" event={"ID":"f2d4dd11-346b-496e-886b-9238eaa91508","Type":"ContainerStarted","Data":"305849b7723e1328a9f737262b8c3d8850a91a93a4fcf508557a68cae9eae5f9"} Jul 05 13:18:57.319056 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:57.319021 2567 generic.go:356] "Generic (PLEG): container finished" podID="d164f387-3a93-465a-a587-ac48c676e2f6" containerID="342e2e236173d12d276c5a90a80d7e5a5dffcb52a09a47b8fe7dc430aafc4f48" exitCode=0 Jul 05 13:18:57.319148 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:57.319063 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk7fc" event={"ID":"d164f387-3a93-465a-a587-ac48c676e2f6","Type":"ContainerDied","Data":"342e2e236173d12d276c5a90a80d7e5a5dffcb52a09a47b8fe7dc430aafc4f48"} Jul 05 13:18:57.329476 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:57.329431 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qmf78" podStartSLOduration=4.124653566 podStartE2EDuration="23.329415902s" totalImagesPullingTime="19.204762336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:18:57.329042512 +0000 UTC m=+23.670060284" watchObservedRunningTime="2026-07-05 13:18:57.329415902 +0000 UTC m=+23.670433690" Jul 05 13:18:57.375468 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:57.375425 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8s4z9" podStartSLOduration=4.168607757 podStartE2EDuration="23.375402847s" totalImagesPullingTime="19.20679509s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:18:57.375311326 +0000 UTC m=+23.716329098" watchObservedRunningTime="2026-07-05 13:18:57.375402847 +0000 UTC m=+23.716420618" Jul 05 13:18:57.375703 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:57.375684 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-78f24" podStartSLOduration=4.144904781 podStartE2EDuration="23.375677992s" totalImagesPullingTime="19.230773211s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:18:57.356582429 +0000 UTC m=+23.697600201" watchObservedRunningTime="2026-07-05 13:18:57.375677992 +0000 UTC m=+23.716695764" Jul 05 13:18:57.406687 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:57.406643 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mhlfp" podStartSLOduration=4.179916414 podStartE2EDuration="23.406630643s" totalImagesPullingTime="19.226714229s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:18:57.406534259 +0000 UTC m=+23.747552030" watchObservedRunningTime="2026-07-05 13:18:57.406630643 +0000 UTC m=+23.747648414" Jul 05 13:18:57.658248 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:57.658208 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qmf78" Jul 05 13:18:57.658922 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:57.658899 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qmf78" Jul 05 13:18:57.683398 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:57.683342 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-k882t" podStartSLOduration=4.413753764 podStartE2EDuration="23.68332483s" totalImagesPullingTime="19.269571066s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:18:57.468857583 +0000 UTC m=+23.809875355" watchObservedRunningTime="2026-07-05 13:18:57.68332483 +0000 UTC m=+24.024342602" Jul 05 13:18:57.866149 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:57.866124 2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 05 13:18:58.140961 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:58.140828 2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-05T13:18:57.866142099Z","UUID":"378db931-c7e0-49b5-9518-2cf254878e8a","Handler":null,"Name":"","Endpoint":""} Jul 05 13:18:58.142639 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:58.142605 2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 05 13:18:58.142799 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:58.142654 2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 05 13:18:58.209981 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:58.209933 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:18:58.210453 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:58.209992 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:18:58.210453 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:58.210065 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j5rvd" podUID="2d509ded-4089-44e1-994f-75d2b4570fce" Jul 05 13:18:58.210453 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:58.210197 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k92vt" podUID="32a4a45d-3b64-476e-9147-09693f809841" Jul 05 13:18:58.324185 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:58.323615 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" event={"ID":"fed1aa39-8ecb-4770-9dea-c73e7b76386a","Type":"ContainerStarted","Data":"05e58467ba9e2404cdfe0253751b89418b4e56342495f0b3db0da7b1b15947ff"} Jul 05 13:18:58.326138 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:58.326103 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xj828" event={"ID":"7364f530-c4ff-47ba-a86b-904542a26d07","Type":"ContainerStarted","Data":"ddb389103c5710aa8d97a34e5f6004feff480b1dbb2b987f08bc91289b0be2ea"} Jul 05 13:18:58.326891 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:58.326859 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qmf78" Jul 05 13:18:58.327147 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:58.327117 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qmf78" Jul 05 13:18:58.348642 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:58.348575 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xj828" podStartSLOduration=5.139024584 podStartE2EDuration="24.348551758s" totalImagesPullingTime="19.209527174s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:18:58.348068305 +0000 UTC m=+24.689086078" watchObservedRunningTime="2026-07-05 13:18:58.348551758 +0000 UTC m=+24.689569532" Jul 05 13:18:59.209550 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:59.209508 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:18:59.209798 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:18:59.209653 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sgqr" podUID="bcbf773f-ee0a-42bd-8983-a1a3bb066e5d" Jul 05 13:18:59.331239 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:59.331003 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" event={"ID":"fed1aa39-8ecb-4770-9dea-c73e7b76386a","Type":"ContainerStarted","Data":"8ba7f8cf7a7f92b3ffb7631562195e32d29dc3d0ffab346e3e054d1d67d1aed4"} Jul 05 13:18:59.336289 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:59.336262 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swckn_a447ce81-b3cb-4269-8d46-e6e612b89e0b/ovn-acl-logging/0.log" Jul 05 13:18:59.336912 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:59.336879 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swckn" event={"ID":"a447ce81-b3cb-4269-8d46-e6e612b89e0b","Type":"ContainerStarted","Data":"2ce3bf7594966bf6c5ba1932297c8c4529904b8ce42f32681c72b45271ace178"} Jul 05 13:18:59.350585 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:18:59.350526 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkcrw" podStartSLOduration=3.84960158 podStartE2EDuration="25.350500696s" totalImagesPullingTime="21.500899116s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:18:59.350071436 +0000 UTC m=+25.691089209" watchObservedRunningTime="2026-07-05 13:18:59.350500696 +0000 UTC m=+25.691518468" Jul 05 13:19:00.209704 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:00.209670 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:19:00.209919 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:00.209829 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k92vt" podUID="32a4a45d-3b64-476e-9147-09693f809841" Jul 05 13:19:00.209919 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:00.209864 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:19:00.210034 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:00.209977 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j5rvd" podUID="2d509ded-4089-44e1-994f-75d2b4570fce" Jul 05 13:19:01.209671 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:01.209631 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:19:01.210125 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:01.209802 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sgqr" podUID="bcbf773f-ee0a-42bd-8983-a1a3bb066e5d" Jul 05 13:19:02.209998 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:02.209816 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:19:02.210784 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:02.209840 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:19:02.210784 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:02.210091 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j5rvd" podUID="2d509ded-4089-44e1-994f-75d2b4570fce" Jul 05 13:19:02.210784 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:02.210131 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k92vt" podUID="32a4a45d-3b64-476e-9147-09693f809841" Jul 05 13:19:02.344592 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:02.344559 2567 generic.go:356] "Generic (PLEG): container finished" podID="d164f387-3a93-465a-a587-ac48c676e2f6" containerID="c5857c5cd9ee202862dbef90b124c82eefb8dcd8df477798bb3217c4af5fc797" exitCode=0 Jul 05 13:19:02.344790 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:02.344656 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk7fc" event={"ID":"d164f387-3a93-465a-a587-ac48c676e2f6","Type":"ContainerDied","Data":"c5857c5cd9ee202862dbef90b124c82eefb8dcd8df477798bb3217c4af5fc797"} Jul 05 13:19:02.348054 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:02.348037 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swckn_a447ce81-b3cb-4269-8d46-e6e612b89e0b/ovn-acl-logging/0.log" Jul 05 13:19:02.348474 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:02.348452 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swckn" event={"ID":"a447ce81-b3cb-4269-8d46-e6e612b89e0b","Type":"ContainerStarted","Data":"9cc3b39e323508d4b7c9114560500f2349040ea0ffceb3b2af0c90773ed6ac18"} Jul 05 13:19:02.348781 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:02.348762 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:19:02.348899 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:02.348877 2567 scope.go:117] "RemoveContainer" containerID="57243dc0d2ebe29ec8c1c7466017cd45827efb4b28f2c089844b5a985c941eee" Jul 05 13:19:02.366087 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:02.366060 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:19:03.209791 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:03.209734 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:19:03.210041 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:03.209868 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sgqr" podUID="bcbf773f-ee0a-42bd-8983-a1a3bb066e5d" Jul 05 13:19:03.355309 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:03.355284 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swckn_a447ce81-b3cb-4269-8d46-e6e612b89e0b/ovn-acl-logging/0.log" Jul 05 13:19:03.355660 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:03.355635 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swckn" event={"ID":"a447ce81-b3cb-4269-8d46-e6e612b89e0b","Type":"ContainerStarted","Data":"5058215350a787105ddf49129e73506931c97a2a7f5529221dff40dbfbce11f9"} Jul 05 13:19:03.355890 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:03.355867 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:19:03.355965 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:03.355949 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:19:03.374120 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:03.374094 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:19:03.386386 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:03.386322 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-swckn" podStartSLOduration=10.095655409 podStartE2EDuration="29.386301561s" totalImagesPullingTime="19.290646152s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:03.385210641 +0000 UTC m=+29.726228415" watchObservedRunningTime="2026-07-05 13:19:03.386301561 +0000 UTC m=+29.727319354" Jul 05 13:19:03.536623 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:03.536587 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k92vt"] Jul 05 13:19:03.536835 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:03.536734 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:19:03.536900 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:03.536865 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k92vt" podUID="32a4a45d-3b64-476e-9147-09693f809841" Jul 05 13:19:03.538757 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:03.538711 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-j5rvd"] Jul 05 13:19:03.538900 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:03.538861 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:19:03.538989 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:03.538966 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j5rvd" podUID="2d509ded-4089-44e1-994f-75d2b4570fce" Jul 05 13:19:03.541540 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:03.541512 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9sgqr"] Jul 05 13:19:03.541670 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:03.541604 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:19:03.541730 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:03.541695 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sgqr" podUID="bcbf773f-ee0a-42bd-8983-a1a3bb066e5d" Jul 05 13:19:04.360159 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:04.360089 2567 generic.go:356] "Generic (PLEG): container finished" podID="d164f387-3a93-465a-a587-ac48c676e2f6" containerID="22e1c1848ad4909235dc910d13a20bb290504031656d238b80ac2ddff458eb45" exitCode=0 Jul 05 13:19:04.360982 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:04.360169 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk7fc" event={"ID":"d164f387-3a93-465a-a587-ac48c676e2f6","Type":"ContainerDied","Data":"22e1c1848ad4909235dc910d13a20bb290504031656d238b80ac2ddff458eb45"} Jul 05 13:19:05.210028 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:05.209979 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:19:05.210201 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:05.209979 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:19:05.210201 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:05.210104 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j5rvd" podUID="2d509ded-4089-44e1-994f-75d2b4570fce" Jul 05 13:19:05.210201 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:05.210008 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:19:05.210201 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:05.210196 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sgqr" podUID="bcbf773f-ee0a-42bd-8983-a1a3bb066e5d" Jul 05 13:19:05.210366 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:05.210269 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k92vt" podUID="32a4a45d-3b64-476e-9147-09693f809841" Jul 05 13:19:06.367279 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:06.367088 2567 generic.go:356] "Generic (PLEG): container finished" podID="d164f387-3a93-465a-a587-ac48c676e2f6" containerID="dc6a48b05e3ca59695b28fde7550e941bc9cab16104023864542d3a130b4a105" exitCode=0 Jul 05 13:19:06.367813 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:06.367170 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk7fc" event={"ID":"d164f387-3a93-465a-a587-ac48c676e2f6","Type":"ContainerDied","Data":"dc6a48b05e3ca59695b28fde7550e941bc9cab16104023864542d3a130b4a105"} Jul 05 13:19:07.210434 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:07.209951 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:19:07.210434 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:07.209987 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:19:07.210434 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:07.210084 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j5rvd" podUID="2d509ded-4089-44e1-994f-75d2b4570fce" Jul 05 13:19:07.210434 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:07.210140 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:19:07.210434 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:07.210238 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sgqr" podUID="bcbf773f-ee0a-42bd-8983-a1a3bb066e5d" Jul 05 13:19:07.210434 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:07.210318 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k92vt" podUID="32a4a45d-3b64-476e-9147-09693f809841" Jul 05 13:19:07.837908 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:07.837874 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-metrics-certs\") pod \"network-metrics-daemon-9sgqr\" (UID: \"bcbf773f-ee0a-42bd-8983-a1a3bb066e5d\") " pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:19:07.837908 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:07.837924 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32a4a45d-3b64-476e-9147-09693f809841-original-pull-secret\") pod \"global-pull-secret-syncer-k92vt\" (UID: \"32a4a45d-3b64-476e-9147-09693f809841\") " pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:19:07.838440 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:07.838034 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:19:07.838440 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:07.838034 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:19:07.838440 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:07.838103 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32a4a45d-3b64-476e-9147-09693f809841-original-pull-secret podName:32a4a45d-3b64-476e-9147-09693f809841 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:39.83808472 +0000 UTC m=+66.179102488 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/32a4a45d-3b64-476e-9147-09693f809841-original-pull-secret") pod "global-pull-secret-syncer-k92vt" (UID: "32a4a45d-3b64-476e-9147-09693f809841") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:19:07.838440 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:07.838126 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-metrics-certs podName:bcbf773f-ee0a-42bd-8983-a1a3bb066e5d nodeName:}" failed. No retries permitted until 2026-07-05 13:19:39.838114176 +0000 UTC m=+66.179131925 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-metrics-certs") pod "network-metrics-daemon-9sgqr" (UID: "bcbf773f-ee0a-42bd-8983-a1a3bb066e5d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:19:07.938773 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:07.938712 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6w76\" (UniqueName: \"kubernetes.io/projected/2d509ded-4089-44e1-994f-75d2b4570fce-kube-api-access-x6w76\") pod \"network-check-target-j5rvd\" (UID: \"2d509ded-4089-44e1-994f-75d2b4570fce\") " pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:19:07.938961 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:07.938935 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:19:07.939019 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:07.938999 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:19:07.939019 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:07.939015 2567 projected.go:196] Error preparing data for projected volume kube-api-access-x6w76 for pod openshift-network-diagnostics/network-check-target-j5rvd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:19:07.939095 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:07.939081 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d509ded-4089-44e1-994f-75d2b4570fce-kube-api-access-x6w76 podName:2d509ded-4089-44e1-994f-75d2b4570fce nodeName:}" failed. No retries permitted until 2026-07-05 13:19:39.939061505 +0000 UTC m=+66.280079266 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-x6w76" (UniqueName: "kubernetes.io/projected/2d509ded-4089-44e1-994f-75d2b4570fce-kube-api-access-x6w76") pod "network-check-target-j5rvd" (UID: "2d509ded-4089-44e1-994f-75d2b4570fce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:19:09.209691 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.209652 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:19:09.210292 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.209650 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:19:09.210292 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:09.209813 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sgqr" podUID="bcbf773f-ee0a-42bd-8983-a1a3bb066e5d" Jul 05 13:19:09.210292 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:09.209871 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j5rvd" podUID="2d509ded-4089-44e1-994f-75d2b4570fce" Jul 05 13:19:09.210292 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.209668 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:19:09.210292 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:09.209987 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k92vt" podUID="32a4a45d-3b64-476e-9147-09693f809841" Jul 05 13:19:09.498326 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.498291 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-119.ec2.internal" event="NodeReady" Jul 05 13:19:09.498528 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.498466 2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 05 13:19:09.536107 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.536070 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7986657c49-hk22t"] Jul 05 13:19:09.539104 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.538527 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:09.541472 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.541442 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 05 13:19:09.541596 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.541488 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 05 13:19:09.541938 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.541918 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 05 13:19:09.543048 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.543028 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jcbpr\"" Jul 05 13:19:09.547090 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.547049 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 05 13:19:09.549064 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.548635 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mkxp5"] Jul 05 13:19:09.550895 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.550840 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8k5ph"] Jul 05 13:19:09.551003 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.550992 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mkxp5" Jul 05 13:19:09.553783 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.553735 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 05 13:19:09.554329 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.554311 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 05 13:19:09.554329 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.554321 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dww7h\"" Jul 05 13:19:09.559536 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.559510 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7986657c49-hk22t"] Jul 05 13:19:09.559718 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.559654 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8k5ph" Jul 05 13:19:09.560712 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.560693 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mkxp5"] Jul 05 13:19:09.562377 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.562354 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 05 13:19:09.562528 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.562511 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 05 13:19:09.562664 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.562646 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 05 13:19:09.562809 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.562764 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gcgw5\"" Jul 05 13:19:09.574731 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.574235 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8k5ph"] Jul 05 13:19:09.651653 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.651611 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f14e6cf-c089-437b-90b6-9cae0be31c56-tmp-dir\") pod \"dns-default-mkxp5\" (UID: \"4f14e6cf-c089-437b-90b6-9cae0be31c56\") " pod="openshift-dns/dns-default-mkxp5" Jul 05 13:19:09.651653 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.651653 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f14e6cf-c089-437b-90b6-9cae0be31c56-metrics-tls\") pod \"dns-default-mkxp5\" (UID: \"4f14e6cf-c089-437b-90b6-9cae0be31c56\") " pod="openshift-dns/dns-default-mkxp5" Jul 05 13:19:09.651910 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.651717 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwn9g\" (UniqueName: \"kubernetes.io/projected/4f14e6cf-c089-437b-90b6-9cae0be31c56-kube-api-access-hwn9g\") pod \"dns-default-mkxp5\" (UID: \"4f14e6cf-c089-437b-90b6-9cae0be31c56\") " pod="openshift-dns/dns-default-mkxp5" Jul 05 13:19:09.651910 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.651783 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b1ce72ed-9388-4861-a28b-4f223346551b-registry-certificates\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:09.651910 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.651822 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-bound-sa-token\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:09.651910 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.651841 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-754wm\" (UniqueName: \"kubernetes.io/projected/99c92f04-b25b-45b6-aa87-f7085cc847eb-kube-api-access-754wm\") pod \"ingress-canary-8k5ph\" (UID: \"99c92f04-b25b-45b6-aa87-f7085cc847eb\") " pod="openshift-ingress-canary/ingress-canary-8k5ph" Jul 05 13:19:09.651910 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.651867 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-registry-tls\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:09.651910 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.651883 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b1ce72ed-9388-4861-a28b-4f223346551b-installation-pull-secrets\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:09.652185 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.651915 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b1ce72ed-9388-4861-a28b-4f223346551b-image-registry-private-configuration\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:09.652185 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.651942 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b1ce72ed-9388-4861-a28b-4f223346551b-trusted-ca\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:09.652185 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.651991 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnbb5\" (UniqueName: \"kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-kube-api-access-mnbb5\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:09.652185 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.652012 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b1ce72ed-9388-4861-a28b-4f223346551b-ca-trust-extracted\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:09.652185 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.652056 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99c92f04-b25b-45b6-aa87-f7085cc847eb-cert\") pod \"ingress-canary-8k5ph\" (UID: \"99c92f04-b25b-45b6-aa87-f7085cc847eb\") " pod="openshift-ingress-canary/ingress-canary-8k5ph" Jul 05 13:19:09.652185 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.652119 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4f14e6cf-c089-437b-90b6-9cae0be31c56-config-volume\") pod \"dns-default-mkxp5\" (UID: \"4f14e6cf-c089-437b-90b6-9cae0be31c56\") " pod="openshift-dns/dns-default-mkxp5" Jul 05 13:19:09.712008 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.711962 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qmf78_df0a5b01-f000-4879-8fcc-175f435bce3e/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:09.752971 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.752877 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4f14e6cf-c089-437b-90b6-9cae0be31c56-config-volume\") pod \"dns-default-mkxp5\" (UID: \"4f14e6cf-c089-437b-90b6-9cae0be31c56\") " pod="openshift-dns/dns-default-mkxp5" Jul 05 13:19:09.752971 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.752927 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f14e6cf-c089-437b-90b6-9cae0be31c56-tmp-dir\") pod \"dns-default-mkxp5\" (UID: \"4f14e6cf-c089-437b-90b6-9cae0be31c56\") " pod="openshift-dns/dns-default-mkxp5" Jul 05 13:19:09.752971 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.752950 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f14e6cf-c089-437b-90b6-9cae0be31c56-metrics-tls\") pod \"dns-default-mkxp5\" (UID: \"4f14e6cf-c089-437b-90b6-9cae0be31c56\") " pod="openshift-dns/dns-default-mkxp5" Jul 05 13:19:09.752971 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.752972 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwn9g\" (UniqueName: \"kubernetes.io/projected/4f14e6cf-c089-437b-90b6-9cae0be31c56-kube-api-access-hwn9g\") pod \"dns-default-mkxp5\" (UID: \"4f14e6cf-c089-437b-90b6-9cae0be31c56\") " pod="openshift-dns/dns-default-mkxp5" Jul 05 13:19:09.753279 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.753006 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b1ce72ed-9388-4861-a28b-4f223346551b-registry-certificates\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:09.753279 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.753077 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-bound-sa-token\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:09.753279 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.753129 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-754wm\" (UniqueName: \"kubernetes.io/projected/99c92f04-b25b-45b6-aa87-f7085cc847eb-kube-api-access-754wm\") pod \"ingress-canary-8k5ph\" (UID: \"99c92f04-b25b-45b6-aa87-f7085cc847eb\") " pod="openshift-ingress-canary/ingress-canary-8k5ph" Jul 05 13:19:09.753279 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.753160 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-registry-tls\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:09.753279 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.753185 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b1ce72ed-9388-4861-a28b-4f223346551b-installation-pull-secrets\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:09.753279 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.753223 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b1ce72ed-9388-4861-a28b-4f223346551b-image-registry-private-configuration\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:09.753279 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.753244 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b1ce72ed-9388-4861-a28b-4f223346551b-trusted-ca\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:09.753603 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.753292 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f14e6cf-c089-437b-90b6-9cae0be31c56-tmp-dir\") pod \"dns-default-mkxp5\" (UID: \"4f14e6cf-c089-437b-90b6-9cae0be31c56\") " pod="openshift-dns/dns-default-mkxp5" Jul 05 13:19:09.753603 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.753306 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnbb5\" (UniqueName: \"kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-kube-api-access-mnbb5\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:09.753603 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.753346 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b1ce72ed-9388-4861-a28b-4f223346551b-ca-trust-extracted\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:09.753603 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.753368 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99c92f04-b25b-45b6-aa87-f7085cc847eb-cert\") pod \"ingress-canary-8k5ph\" (UID: \"99c92f04-b25b-45b6-aa87-f7085cc847eb\") " pod="openshift-ingress-canary/ingress-canary-8k5ph" Jul 05 13:19:09.753603 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:09.753383 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:19:09.753603 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:09.753439 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f14e6cf-c089-437b-90b6-9cae0be31c56-metrics-tls podName:4f14e6cf-c089-437b-90b6-9cae0be31c56 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:10.253421712 +0000 UTC m=+36.594439462 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4f14e6cf-c089-437b-90b6-9cae0be31c56-metrics-tls") pod "dns-default-mkxp5" (UID: "4f14e6cf-c089-437b-90b6-9cae0be31c56") : secret "dns-default-metrics-tls" not found Jul 05 13:19:09.753920 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.753618 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b1ce72ed-9388-4861-a28b-4f223346551b-registry-certificates\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:09.753920 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.753623 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4f14e6cf-c089-437b-90b6-9cae0be31c56-config-volume\") pod \"dns-default-mkxp5\" (UID: \"4f14e6cf-c089-437b-90b6-9cae0be31c56\") " pod="openshift-dns/dns-default-mkxp5" Jul 05 13:19:09.754151 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:09.754134 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:19:09.754151 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:09.754153 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7986657c49-hk22t: secret "image-registry-tls" not found Jul 05 13:19:09.754302 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.754158 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b1ce72ed-9388-4861-a28b-4f223346551b-ca-trust-extracted\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:09.754302 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:09.754204 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-registry-tls podName:b1ce72ed-9388-4861-a28b-4f223346551b nodeName:}" failed. No retries permitted until 2026-07-05 13:19:10.254188706 +0000 UTC m=+36.595206470 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-registry-tls") pod "image-registry-7986657c49-hk22t" (UID: "b1ce72ed-9388-4861-a28b-4f223346551b") : secret "image-registry-tls" not found Jul 05 13:19:09.754582 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.754560 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b1ce72ed-9388-4861-a28b-4f223346551b-trusted-ca\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:09.758430 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.758400 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b1ce72ed-9388-4861-a28b-4f223346551b-installation-pull-secrets\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:09.758534 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.758443 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b1ce72ed-9388-4861-a28b-4f223346551b-image-registry-private-configuration\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:09.758534 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.758463 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99c92f04-b25b-45b6-aa87-f7085cc847eb-cert\") pod \"ingress-canary-8k5ph\" (UID: \"99c92f04-b25b-45b6-aa87-f7085cc847eb\") " pod="openshift-ingress-canary/ingress-canary-8k5ph" Jul 05 13:19:09.762410 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.762375 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-754wm\" (UniqueName: \"kubernetes.io/projected/99c92f04-b25b-45b6-aa87-f7085cc847eb-kube-api-access-754wm\") pod \"ingress-canary-8k5ph\" (UID: \"99c92f04-b25b-45b6-aa87-f7085cc847eb\") " pod="openshift-ingress-canary/ingress-canary-8k5ph" Jul 05 13:19:09.762631 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.762607 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnbb5\" (UniqueName: \"kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-kube-api-access-mnbb5\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:09.762773 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.762629 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwn9g\" (UniqueName: \"kubernetes.io/projected/4f14e6cf-c089-437b-90b6-9cae0be31c56-kube-api-access-hwn9g\") pod \"dns-default-mkxp5\" (UID: \"4f14e6cf-c089-437b-90b6-9cae0be31c56\") " pod="openshift-dns/dns-default-mkxp5" Jul 05 13:19:09.763229 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.763209 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-bound-sa-token\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:09.802682 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.802645 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qmf78_df0a5b01-f000-4879-8fcc-175f435bce3e/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:09.875863 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:09.875819 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8k5ph" Jul 05 13:19:10.256265 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:10.256222 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f14e6cf-c089-437b-90b6-9cae0be31c56-metrics-tls\") pod \"dns-default-mkxp5\" (UID: \"4f14e6cf-c089-437b-90b6-9cae0be31c56\") " pod="openshift-dns/dns-default-mkxp5" Jul 05 13:19:10.256947 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:10.256300 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-registry-tls\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:10.256947 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:10.256427 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:19:10.256947 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:10.256465 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:19:10.256947 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:10.256485 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7986657c49-hk22t: secret "image-registry-tls" not found Jul 05 13:19:10.256947 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:10.256490 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f14e6cf-c089-437b-90b6-9cae0be31c56-metrics-tls podName:4f14e6cf-c089-437b-90b6-9cae0be31c56 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:11.256473179 +0000 UTC m=+37.597490932 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4f14e6cf-c089-437b-90b6-9cae0be31c56-metrics-tls") pod "dns-default-mkxp5" (UID: "4f14e6cf-c089-437b-90b6-9cae0be31c56") : secret "dns-default-metrics-tls" not found Jul 05 13:19:10.256947 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:10.256550 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-registry-tls podName:b1ce72ed-9388-4861-a28b-4f223346551b nodeName:}" failed. No retries permitted until 2026-07-05 13:19:11.256533501 +0000 UTC m=+37.597551252 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-registry-tls") pod "image-registry-7986657c49-hk22t" (UID: "b1ce72ed-9388-4861-a28b-4f223346551b") : secret "image-registry-tls" not found Jul 05 13:19:10.657138 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:10.657108 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8k5ph"] Jul 05 13:19:10.660878 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:19:10.660853 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99c92f04_b25b_45b6_aa87_f7085cc847eb.slice/crio-842e2ef1b197407dd92d67898fd7331d6e03a4899d7047f55785ec25f84d636e WatchSource:0}: Error finding container 842e2ef1b197407dd92d67898fd7331d6e03a4899d7047f55785ec25f84d636e: Status 404 returned error can't find the container with id 842e2ef1b197407dd92d67898fd7331d6e03a4899d7047f55785ec25f84d636e Jul 05 13:19:11.209665 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:11.209617 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:19:11.209665 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:11.209648 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:19:11.209938 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:11.209658 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:19:11.212716 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:11.212682 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 05 13:19:11.212877 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:11.212762 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 05 13:19:11.213997 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:11.213975 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 05 13:19:11.213997 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:11.213987 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6n5k8\"" Jul 05 13:19:11.214195 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:11.214053 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 05 13:19:11.214195 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:11.213990 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kn7g2\"" Jul 05 13:19:11.265721 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:11.265673 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f14e6cf-c089-437b-90b6-9cae0be31c56-metrics-tls\") pod \"dns-default-mkxp5\" (UID: \"4f14e6cf-c089-437b-90b6-9cae0be31c56\") " pod="openshift-dns/dns-default-mkxp5" Jul 05 13:19:11.266183 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:11.265757 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-registry-tls\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:11.266183 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:11.265904 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:19:11.266183 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:11.265919 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7986657c49-hk22t: secret "image-registry-tls" not found Jul 05 13:19:11.266183 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:11.266010 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-registry-tls podName:b1ce72ed-9388-4861-a28b-4f223346551b nodeName:}" failed. No retries permitted until 2026-07-05 13:19:13.265990645 +0000 UTC m=+39.607008408 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-registry-tls") pod "image-registry-7986657c49-hk22t" (UID: "b1ce72ed-9388-4861-a28b-4f223346551b") : secret "image-registry-tls" not found Jul 05 13:19:11.266442 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:11.266425 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:19:11.266516 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:11.266479 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f14e6cf-c089-437b-90b6-9cae0be31c56-metrics-tls podName:4f14e6cf-c089-437b-90b6-9cae0be31c56 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:13.266465523 +0000 UTC m=+39.607483281 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4f14e6cf-c089-437b-90b6-9cae0be31c56-metrics-tls") pod "dns-default-mkxp5" (UID: "4f14e6cf-c089-437b-90b6-9cae0be31c56") : secret "dns-default-metrics-tls" not found Jul 05 13:19:11.383342 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:11.383302 2567 generic.go:356] "Generic (PLEG): container finished" podID="d164f387-3a93-465a-a587-ac48c676e2f6" containerID="96a6459846937bd48dee8ef3ef1ec2fccc4a9bca86d958ced3aded0f9e9c4767" exitCode=0 Jul 05 13:19:11.383519 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:11.383376 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk7fc" event={"ID":"d164f387-3a93-465a-a587-ac48c676e2f6","Type":"ContainerDied","Data":"96a6459846937bd48dee8ef3ef1ec2fccc4a9bca86d958ced3aded0f9e9c4767"} Jul 05 13:19:11.384608 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:11.384579 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8k5ph" event={"ID":"99c92f04-b25b-45b6-aa87-f7085cc847eb","Type":"ContainerStarted","Data":"842e2ef1b197407dd92d67898fd7331d6e03a4899d7047f55785ec25f84d636e"} Jul 05 13:19:12.392094 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:12.392059 2567 generic.go:356] "Generic (PLEG): container finished" podID="d164f387-3a93-465a-a587-ac48c676e2f6" containerID="48a7b6c0a7425dcf9bd0b121b43f4bd77c877fea33eef49c7ae7ee412aa90caf" exitCode=0 Jul 05 13:19:12.392587 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:12.392114 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk7fc" event={"ID":"d164f387-3a93-465a-a587-ac48c676e2f6","Type":"ContainerDied","Data":"48a7b6c0a7425dcf9bd0b121b43f4bd77c877fea33eef49c7ae7ee412aa90caf"} Jul 05 13:19:13.281635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:13.281410 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f14e6cf-c089-437b-90b6-9cae0be31c56-metrics-tls\") pod \"dns-default-mkxp5\" (UID: \"4f14e6cf-c089-437b-90b6-9cae0be31c56\") " pod="openshift-dns/dns-default-mkxp5" Jul 05 13:19:13.281635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:13.281611 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-registry-tls\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:13.281853 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:13.281557 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:19:13.281853 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:13.281701 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f14e6cf-c089-437b-90b6-9cae0be31c56-metrics-tls podName:4f14e6cf-c089-437b-90b6-9cae0be31c56 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:17.281686784 +0000 UTC m=+43.622704535 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4f14e6cf-c089-437b-90b6-9cae0be31c56-metrics-tls") pod "dns-default-mkxp5" (UID: "4f14e6cf-c089-437b-90b6-9cae0be31c56") : secret "dns-default-metrics-tls" not found Jul 05 13:19:13.281853 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:13.281714 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:19:13.281853 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:13.281725 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7986657c49-hk22t: secret "image-registry-tls" not found Jul 05 13:19:13.281853 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:13.281803 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-registry-tls podName:b1ce72ed-9388-4861-a28b-4f223346551b nodeName:}" failed. No retries permitted until 2026-07-05 13:19:17.281791183 +0000 UTC m=+43.622808934 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-registry-tls") pod "image-registry-7986657c49-hk22t" (UID: "b1ce72ed-9388-4861-a28b-4f223346551b") : secret "image-registry-tls" not found Jul 05 13:19:13.397883 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:13.397850 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk7fc" event={"ID":"d164f387-3a93-465a-a587-ac48c676e2f6","Type":"ContainerStarted","Data":"ef05813c9f7a03b225646b509aa25865b0c3f1aa0d641d4fe578a4b0cf6548fe"} Jul 05 13:19:13.399250 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:13.399226 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8k5ph" event={"ID":"99c92f04-b25b-45b6-aa87-f7085cc847eb","Type":"ContainerStarted","Data":"32dc04eaf43f6fbb2c8150db9aa63882f6cdb016d37bbed3b9f97760fbee3733"} Jul 05 13:19:13.421368 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:13.421313 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wk7fc" podStartSLOduration=9.533539526 podStartE2EDuration="39.421296866s" totalImagesPullingTime="29.88775734s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:34 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:13.419882253 +0000 UTC m=+39.760900024" watchObservedRunningTime="2026-07-05 13:19:13.421296866 +0000 UTC m=+39.762314638" Jul 05 13:19:13.435705 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:13.435649 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8k5ph" podStartSLOduration=2.23736379 podStartE2EDuration="4.435634379s" totalImagesPullingTime="2.198270589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:13.43499252 +0000 UTC m=+39.776010292" watchObservedRunningTime="2026-07-05 13:19:13.435634379 +0000 UTC m=+39.776652151" Jul 05 13:19:14.963283 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:14.963244 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7tlcm"] Jul 05 13:19:14.992566 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:14.992513 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7tlcm"] Jul 05 13:19:14.992728 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:14.992624 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7tlcm" Jul 05 13:19:14.995285 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:14.995252 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 05 13:19:14.996361 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:14.996339 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 05 13:19:14.996361 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:14.996357 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-4fg58\"" Jul 05 13:19:14.996552 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:14.996343 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:19:14.996552 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:14.996379 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 05 13:19:15.071115 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.071077 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-67485"] Jul 05 13:19:15.093663 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.093629 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/98016b6f-895f-4fe5-9607-1d2c7ec2f8a3-config\") pod \"service-ca-operator-5d46d9ffd4-7tlcm\" (UID: \"98016b6f-895f-4fe5-9607-1d2c7ec2f8a3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7tlcm" Jul 05 13:19:15.093917 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.093886 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/98016b6f-895f-4fe5-9607-1d2c7ec2f8a3-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-7tlcm\" (UID: \"98016b6f-895f-4fe5-9607-1d2c7ec2f8a3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7tlcm" Jul 05 13:19:15.094047 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.094014 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwb7p\" (UniqueName: \"kubernetes.io/projected/98016b6f-895f-4fe5-9607-1d2c7ec2f8a3-kube-api-access-jwb7p\") pod \"service-ca-operator-5d46d9ffd4-7tlcm\" (UID: \"98016b6f-895f-4fe5-9607-1d2c7ec2f8a3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7tlcm" Jul 05 13:19:15.094580 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.094560 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8dj7d"] Jul 05 13:19:15.094686 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.094672 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-67485" Jul 05 13:19:15.097426 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.097408 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-ftgv5\"" Jul 05 13:19:15.116235 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.116194 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-xnqg7"] Jul 05 13:19:15.116402 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.116308 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8dj7d" Jul 05 13:19:15.119006 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.118983 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 05 13:19:15.119134 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.118983 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-pxqg5\"" Jul 05 13:19:15.119134 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.119072 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:19:15.134300 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.134265 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-7vrt7"] Jul 05 13:19:15.134459 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.134395 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-xnqg7" Jul 05 13:19:15.137199 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.137177 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:19:15.137341 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.137217 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 05 13:19:15.137341 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.137217 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-7wcx9\"" Jul 05 13:19:15.137470 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.137447 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 05 13:19:15.148383 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.148358 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-f8jvx"] Jul 05 13:19:15.148527 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.148485 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-7vrt7" Jul 05 13:19:15.151635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.151608 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 05 13:19:15.151814 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.151800 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 05 13:19:15.151869 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.151801 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 05 13:19:15.152330 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.152316 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-sm52h\"" Jul 05 13:19:15.152614 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.152598 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 05 13:19:15.160122 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.160091 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 05 13:19:15.164582 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.164553 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-67485"] Jul 05 13:19:15.164582 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.164591 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8dj7d"] Jul 05 13:19:15.164770 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.164609 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-7vrt7"] Jul 05 13:19:15.164770 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.164624 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-xnqg7"] Jul 05 13:19:15.164770 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.164640 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-f8jvx"] Jul 05 13:19:15.164770 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.164664 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-f8jvx" Jul 05 13:19:15.168657 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.168634 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 05 13:19:15.168854 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.168834 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 05 13:19:15.168908 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.168875 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 05 13:19:15.168962 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.168913 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-kskdk\"" Jul 05 13:19:15.168962 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.168875 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 05 13:19:15.177166 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.177140 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rgmdx"] Jul 05 13:19:15.194564 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.194533 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwb7p\" (UniqueName: \"kubernetes.io/projected/98016b6f-895f-4fe5-9607-1d2c7ec2f8a3-kube-api-access-jwb7p\") pod \"service-ca-operator-5d46d9ffd4-7tlcm\" (UID: \"98016b6f-895f-4fe5-9607-1d2c7ec2f8a3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7tlcm" Jul 05 13:19:15.194564 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.194578 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7cqr\" (UniqueName: \"kubernetes.io/projected/011559fd-ed92-41fe-acae-079a7650cd20-kube-api-access-m7cqr\") pod \"network-check-source-989f8b8dc-67485\" (UID: \"011559fd-ed92-41fe-acae-079a7650cd20\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-67485" Jul 05 13:19:15.194873 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.194680 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/98016b6f-895f-4fe5-9607-1d2c7ec2f8a3-config\") pod \"service-ca-operator-5d46d9ffd4-7tlcm\" (UID: \"98016b6f-895f-4fe5-9607-1d2c7ec2f8a3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7tlcm" Jul 05 13:19:15.194873 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.194712 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/98016b6f-895f-4fe5-9607-1d2c7ec2f8a3-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-7tlcm\" (UID: \"98016b6f-895f-4fe5-9607-1d2c7ec2f8a3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7tlcm" Jul 05 13:19:15.195407 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.195381 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/98016b6f-895f-4fe5-9607-1d2c7ec2f8a3-config\") pod \"service-ca-operator-5d46d9ffd4-7tlcm\" (UID: \"98016b6f-895f-4fe5-9607-1d2c7ec2f8a3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7tlcm" Jul 05 13:19:15.198699 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.198676 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-26jcf"] Jul 05 13:19:15.198832 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.198804 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rgmdx" Jul 05 13:19:15.200037 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.200011 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/98016b6f-895f-4fe5-9607-1d2c7ec2f8a3-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-7tlcm\" (UID: \"98016b6f-895f-4fe5-9607-1d2c7ec2f8a3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7tlcm" Jul 05 13:19:15.203588 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.203500 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 05 13:19:15.203709 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.203639 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:19:15.203938 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.203888 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 05 13:19:15.204011 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.203961 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 05 13:19:15.204084 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.203994 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-7fvft\"" Jul 05 13:19:15.213995 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.213932 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwb7p\" (UniqueName: \"kubernetes.io/projected/98016b6f-895f-4fe5-9607-1d2c7ec2f8a3-kube-api-access-jwb7p\") pod \"service-ca-operator-5d46d9ffd4-7tlcm\" (UID: \"98016b6f-895f-4fe5-9607-1d2c7ec2f8a3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7tlcm" Jul 05 13:19:15.218876 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.218853 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rgmdx"] Jul 05 13:19:15.218958 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.218891 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-644877d48f-jctfg"] Jul 05 13:19:15.219045 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.218963 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-26jcf" Jul 05 13:19:15.221675 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.221656 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 05 13:19:15.221675 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.221672 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:19:15.221849 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.221771 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-c2n46\"" Jul 05 13:19:15.221983 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.221966 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 05 13:19:15.222120 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.222101 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 05 13:19:15.227926 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.227907 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 05 13:19:15.241014 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.240966 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-26jcf"] Jul 05 13:19:15.241014 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.240997 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:15.241276 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.241036 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-644877d48f-jctfg"] Jul 05 13:19:15.243602 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.243569 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 05 13:19:15.243830 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.243815 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-hsq9p\"" Jul 05 13:19:15.243830 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.243826 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 05 13:19:15.243984 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.243878 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 05 13:19:15.244081 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.244068 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 05 13:19:15.244260 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.244241 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 05 13:19:15.244314 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.244265 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 05 13:19:15.295576 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.295541 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/352ab893-ca08-40ef-9a02-583440eb4788-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-f8jvx\" (UID: \"352ab893-ca08-40ef-9a02-583440eb4788\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-f8jvx" Jul 05 13:19:15.295576 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.295589 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2dfp\" (UniqueName: \"kubernetes.io/projected/532a0da4-c9ae-4e33-96e8-381c6715d7e9-kube-api-access-j2dfp\") pod \"insights-operator-5c847dcfc4-7vrt7\" (UID: \"532a0da4-c9ae-4e33-96e8-381c6715d7e9\") " pod="openshift-insights/insights-operator-5c847dcfc4-7vrt7" Jul 05 13:19:15.295875 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.295618 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svc7w\" (UniqueName: \"kubernetes.io/projected/fab50c14-6777-44c0-a7b7-e3b52ea810ca-kube-api-access-svc7w\") pod \"volume-data-source-validator-5d456fdbcd-8dj7d\" (UID: \"fab50c14-6777-44c0-a7b7-e3b52ea810ca\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8dj7d" Jul 05 13:19:15.295875 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.295677 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e21b2ebc-6b56-45d0-ab65-e144505dd4b4-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-rgmdx\" (UID: \"e21b2ebc-6b56-45d0-ab65-e144505dd4b4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rgmdx" Jul 05 13:19:15.295875 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.295701 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/532a0da4-c9ae-4e33-96e8-381c6715d7e9-serving-cert\") pod \"insights-operator-5c847dcfc4-7vrt7\" (UID: \"532a0da4-c9ae-4e33-96e8-381c6715d7e9\") " pod="openshift-insights/insights-operator-5c847dcfc4-7vrt7" Jul 05 13:19:15.295875 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.295761 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdc1a97f-d106-47bf-92ab-967e6af9ef16-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-xnqg7\" (UID: \"cdc1a97f-d106-47bf-92ab-967e6af9ef16\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-xnqg7" Jul 05 13:19:15.295875 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.295796 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e21b2ebc-6b56-45d0-ab65-e144505dd4b4-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-rgmdx\" (UID: \"e21b2ebc-6b56-45d0-ab65-e144505dd4b4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rgmdx" Jul 05 13:19:15.295875 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.295851 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96v2r\" (UniqueName: \"kubernetes.io/projected/352ab893-ca08-40ef-9a02-583440eb4788-kube-api-access-96v2r\") pod \"cluster-monitoring-operator-64786d9cbb-f8jvx\" (UID: \"352ab893-ca08-40ef-9a02-583440eb4788\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-f8jvx" Jul 05 13:19:15.296092 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.295895 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/532a0da4-c9ae-4e33-96e8-381c6715d7e9-tmp\") pod \"insights-operator-5c847dcfc4-7vrt7\" (UID: \"532a0da4-c9ae-4e33-96e8-381c6715d7e9\") " pod="openshift-insights/insights-operator-5c847dcfc4-7vrt7" Jul 05 13:19:15.296092 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.295913 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/532a0da4-c9ae-4e33-96e8-381c6715d7e9-snapshots\") pod \"insights-operator-5c847dcfc4-7vrt7\" (UID: \"532a0da4-c9ae-4e33-96e8-381c6715d7e9\") " pod="openshift-insights/insights-operator-5c847dcfc4-7vrt7" Jul 05 13:19:15.296092 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.295957 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/532a0da4-c9ae-4e33-96e8-381c6715d7e9-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-7vrt7\" (UID: \"532a0da4-c9ae-4e33-96e8-381c6715d7e9\") " pod="openshift-insights/insights-operator-5c847dcfc4-7vrt7" Jul 05 13:19:15.296092 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.296019 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/532a0da4-c9ae-4e33-96e8-381c6715d7e9-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-7vrt7\" (UID: \"532a0da4-c9ae-4e33-96e8-381c6715d7e9\") " pod="openshift-insights/insights-operator-5c847dcfc4-7vrt7" Jul 05 13:19:15.296092 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.296041 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/352ab893-ca08-40ef-9a02-583440eb4788-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-f8jvx\" (UID: \"352ab893-ca08-40ef-9a02-583440eb4788\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-f8jvx" Jul 05 13:19:15.296092 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.296079 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grcxl\" (UniqueName: \"kubernetes.io/projected/cdc1a97f-d106-47bf-92ab-967e6af9ef16-kube-api-access-grcxl\") pod \"cluster-samples-operator-56fcf86bf-xnqg7\" (UID: \"cdc1a97f-d106-47bf-92ab-967e6af9ef16\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-xnqg7" Jul 05 13:19:15.296334 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.296135 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7cqr\" (UniqueName: \"kubernetes.io/projected/011559fd-ed92-41fe-acae-079a7650cd20-kube-api-access-m7cqr\") pod \"network-check-source-989f8b8dc-67485\" (UID: \"011559fd-ed92-41fe-acae-079a7650cd20\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-67485" Jul 05 13:19:15.296334 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.296171 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jw2x\" (UniqueName: \"kubernetes.io/projected/e21b2ebc-6b56-45d0-ab65-e144505dd4b4-kube-api-access-2jw2x\") pod \"kube-storage-version-migrator-operator-846bf668f7-rgmdx\" (UID: \"e21b2ebc-6b56-45d0-ab65-e144505dd4b4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rgmdx" Jul 05 13:19:15.301811 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.301794 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7tlcm" Jul 05 13:19:15.304309 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.304290 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7cqr\" (UniqueName: \"kubernetes.io/projected/011559fd-ed92-41fe-acae-079a7650cd20-kube-api-access-m7cqr\") pod \"network-check-source-989f8b8dc-67485\" (UID: \"011559fd-ed92-41fe-acae-079a7650cd20\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-67485" Jul 05 13:19:15.397120 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397083 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdc1a97f-d106-47bf-92ab-967e6af9ef16-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-xnqg7\" (UID: \"cdc1a97f-d106-47bf-92ab-967e6af9ef16\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-xnqg7" Jul 05 13:19:15.397120 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397124 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95b0d0a1-07c4-4235-b8d0-5998a2d26473-service-ca-bundle\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:15.397360 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397146 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e21b2ebc-6b56-45d0-ab65-e144505dd4b4-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-rgmdx\" (UID: \"e21b2ebc-6b56-45d0-ab65-e144505dd4b4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rgmdx" Jul 05 13:19:15.397360 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397166 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96v2r\" (UniqueName: \"kubernetes.io/projected/352ab893-ca08-40ef-9a02-583440eb4788-kube-api-access-96v2r\") pod \"cluster-monitoring-operator-64786d9cbb-f8jvx\" (UID: \"352ab893-ca08-40ef-9a02-583440eb4788\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-f8jvx" Jul 05 13:19:15.397360 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397196 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bccc5e19-26b8-4f6b-96f2-92684dc3d25a-serving-cert\") pod \"console-operator-68b757fb67-26jcf\" (UID: \"bccc5e19-26b8-4f6b-96f2-92684dc3d25a\") " pod="openshift-console-operator/console-operator-68b757fb67-26jcf" Jul 05 13:19:15.397360 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397212 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/532a0da4-c9ae-4e33-96e8-381c6715d7e9-tmp\") pod \"insights-operator-5c847dcfc4-7vrt7\" (UID: \"532a0da4-c9ae-4e33-96e8-381c6715d7e9\") " pod="openshift-insights/insights-operator-5c847dcfc4-7vrt7" Jul 05 13:19:15.397360 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397228 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-stats-auth\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:15.397360 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397249 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/532a0da4-c9ae-4e33-96e8-381c6715d7e9-snapshots\") pod \"insights-operator-5c847dcfc4-7vrt7\" (UID: \"532a0da4-c9ae-4e33-96e8-381c6715d7e9\") " pod="openshift-insights/insights-operator-5c847dcfc4-7vrt7" Jul 05 13:19:15.397360 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:15.397255 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 05 13:19:15.397360 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397272 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/532a0da4-c9ae-4e33-96e8-381c6715d7e9-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-7vrt7\" (UID: \"532a0da4-c9ae-4e33-96e8-381c6715d7e9\") " pod="openshift-insights/insights-operator-5c847dcfc4-7vrt7" Jul 05 13:19:15.397360 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397349 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-metrics-certs\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:15.397856 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:15.397400 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdc1a97f-d106-47bf-92ab-967e6af9ef16-samples-operator-tls podName:cdc1a97f-d106-47bf-92ab-967e6af9ef16 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:15.897374019 +0000 UTC m=+42.238391778 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cdc1a97f-d106-47bf-92ab-967e6af9ef16-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-xnqg7" (UID: "cdc1a97f-d106-47bf-92ab-967e6af9ef16") : secret "samples-operator-tls" not found Jul 05 13:19:15.397856 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397447 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bccc5e19-26b8-4f6b-96f2-92684dc3d25a-config\") pod \"console-operator-68b757fb67-26jcf\" (UID: \"bccc5e19-26b8-4f6b-96f2-92684dc3d25a\") " pod="openshift-console-operator/console-operator-68b757fb67-26jcf" Jul 05 13:19:15.397856 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397475 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j56f\" (UniqueName: \"kubernetes.io/projected/95b0d0a1-07c4-4235-b8d0-5998a2d26473-kube-api-access-7j56f\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:15.397856 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397525 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/532a0da4-c9ae-4e33-96e8-381c6715d7e9-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-7vrt7\" (UID: \"532a0da4-c9ae-4e33-96e8-381c6715d7e9\") " pod="openshift-insights/insights-operator-5c847dcfc4-7vrt7" Jul 05 13:19:15.397856 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397551 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/352ab893-ca08-40ef-9a02-583440eb4788-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-f8jvx\" (UID: \"352ab893-ca08-40ef-9a02-583440eb4788\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-f8jvx" Jul 05 13:19:15.397856 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397555 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/532a0da4-c9ae-4e33-96e8-381c6715d7e9-tmp\") pod \"insights-operator-5c847dcfc4-7vrt7\" (UID: \"532a0da4-c9ae-4e33-96e8-381c6715d7e9\") " pod="openshift-insights/insights-operator-5c847dcfc4-7vrt7" Jul 05 13:19:15.397856 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397618 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grcxl\" (UniqueName: \"kubernetes.io/projected/cdc1a97f-d106-47bf-92ab-967e6af9ef16-kube-api-access-grcxl\") pod \"cluster-samples-operator-56fcf86bf-xnqg7\" (UID: \"cdc1a97f-d106-47bf-92ab-967e6af9ef16\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-xnqg7" Jul 05 13:19:15.397856 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397659 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jw2x\" (UniqueName: \"kubernetes.io/projected/e21b2ebc-6b56-45d0-ab65-e144505dd4b4-kube-api-access-2jw2x\") pod \"kube-storage-version-migrator-operator-846bf668f7-rgmdx\" (UID: \"e21b2ebc-6b56-45d0-ab65-e144505dd4b4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rgmdx" Jul 05 13:19:15.397856 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397726 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/352ab893-ca08-40ef-9a02-583440eb4788-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-f8jvx\" (UID: \"352ab893-ca08-40ef-9a02-583440eb4788\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-f8jvx" Jul 05 13:19:15.397856 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397780 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdlkd\" (UniqueName: \"kubernetes.io/projected/bccc5e19-26b8-4f6b-96f2-92684dc3d25a-kube-api-access-mdlkd\") pod \"console-operator-68b757fb67-26jcf\" (UID: \"bccc5e19-26b8-4f6b-96f2-92684dc3d25a\") " pod="openshift-console-operator/console-operator-68b757fb67-26jcf" Jul 05 13:19:15.397856 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397805 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-default-certificate\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:15.397856 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397840 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2dfp\" (UniqueName: \"kubernetes.io/projected/532a0da4-c9ae-4e33-96e8-381c6715d7e9-kube-api-access-j2dfp\") pod \"insights-operator-5c847dcfc4-7vrt7\" (UID: \"532a0da4-c9ae-4e33-96e8-381c6715d7e9\") " pod="openshift-insights/insights-operator-5c847dcfc4-7vrt7" Jul 05 13:19:15.398470 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397866 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svc7w\" (UniqueName: \"kubernetes.io/projected/fab50c14-6777-44c0-a7b7-e3b52ea810ca-kube-api-access-svc7w\") pod \"volume-data-source-validator-5d456fdbcd-8dj7d\" (UID: \"fab50c14-6777-44c0-a7b7-e3b52ea810ca\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8dj7d" Jul 05 13:19:15.398470 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397873 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e21b2ebc-6b56-45d0-ab65-e144505dd4b4-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-rgmdx\" (UID: \"e21b2ebc-6b56-45d0-ab65-e144505dd4b4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rgmdx" Jul 05 13:19:15.398470 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397918 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e21b2ebc-6b56-45d0-ab65-e144505dd4b4-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-rgmdx\" (UID: \"e21b2ebc-6b56-45d0-ab65-e144505dd4b4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rgmdx" Jul 05 13:19:15.398470 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397945 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bccc5e19-26b8-4f6b-96f2-92684dc3d25a-trusted-ca\") pod \"console-operator-68b757fb67-26jcf\" (UID: \"bccc5e19-26b8-4f6b-96f2-92684dc3d25a\") " pod="openshift-console-operator/console-operator-68b757fb67-26jcf" Jul 05 13:19:15.398470 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.397972 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/532a0da4-c9ae-4e33-96e8-381c6715d7e9-serving-cert\") pod \"insights-operator-5c847dcfc4-7vrt7\" (UID: \"532a0da4-c9ae-4e33-96e8-381c6715d7e9\") " pod="openshift-insights/insights-operator-5c847dcfc4-7vrt7" Jul 05 13:19:15.398470 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:15.397978 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 05 13:19:15.398470 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:15.398036 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/352ab893-ca08-40ef-9a02-583440eb4788-cluster-monitoring-operator-tls podName:352ab893-ca08-40ef-9a02-583440eb4788 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:15.898020213 +0000 UTC m=+42.239037963 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/352ab893-ca08-40ef-9a02-583440eb4788-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-f8jvx" (UID: "352ab893-ca08-40ef-9a02-583440eb4788") : secret "cluster-monitoring-operator-tls" not found Jul 05 13:19:15.398470 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.398170 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/532a0da4-c9ae-4e33-96e8-381c6715d7e9-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-7vrt7\" (UID: \"532a0da4-c9ae-4e33-96e8-381c6715d7e9\") " pod="openshift-insights/insights-operator-5c847dcfc4-7vrt7" Jul 05 13:19:15.398470 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.398351 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/352ab893-ca08-40ef-9a02-583440eb4788-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-f8jvx\" (UID: \"352ab893-ca08-40ef-9a02-583440eb4788\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-f8jvx" Jul 05 13:19:15.398920 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.398651 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/532a0da4-c9ae-4e33-96e8-381c6715d7e9-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-7vrt7\" (UID: \"532a0da4-c9ae-4e33-96e8-381c6715d7e9\") " pod="openshift-insights/insights-operator-5c847dcfc4-7vrt7" Jul 05 13:19:15.398920 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.398709 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/532a0da4-c9ae-4e33-96e8-381c6715d7e9-snapshots\") pod \"insights-operator-5c847dcfc4-7vrt7\" (UID: \"532a0da4-c9ae-4e33-96e8-381c6715d7e9\") " pod="openshift-insights/insights-operator-5c847dcfc4-7vrt7" Jul 05 13:19:15.401056 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.401033 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e21b2ebc-6b56-45d0-ab65-e144505dd4b4-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-rgmdx\" (UID: \"e21b2ebc-6b56-45d0-ab65-e144505dd4b4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rgmdx" Jul 05 13:19:15.401191 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.401118 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/532a0da4-c9ae-4e33-96e8-381c6715d7e9-serving-cert\") pod \"insights-operator-5c847dcfc4-7vrt7\" (UID: \"532a0da4-c9ae-4e33-96e8-381c6715d7e9\") " pod="openshift-insights/insights-operator-5c847dcfc4-7vrt7" Jul 05 13:19:15.403846 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.403822 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-67485" Jul 05 13:19:15.406644 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.406310 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jw2x\" (UniqueName: \"kubernetes.io/projected/e21b2ebc-6b56-45d0-ab65-e144505dd4b4-kube-api-access-2jw2x\") pod \"kube-storage-version-migrator-operator-846bf668f7-rgmdx\" (UID: \"e21b2ebc-6b56-45d0-ab65-e144505dd4b4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rgmdx" Jul 05 13:19:15.408433 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.407485 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grcxl\" (UniqueName: \"kubernetes.io/projected/cdc1a97f-d106-47bf-92ab-967e6af9ef16-kube-api-access-grcxl\") pod \"cluster-samples-operator-56fcf86bf-xnqg7\" (UID: \"cdc1a97f-d106-47bf-92ab-967e6af9ef16\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-xnqg7" Jul 05 13:19:15.408433 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.407617 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96v2r\" (UniqueName: \"kubernetes.io/projected/352ab893-ca08-40ef-9a02-583440eb4788-kube-api-access-96v2r\") pod \"cluster-monitoring-operator-64786d9cbb-f8jvx\" (UID: \"352ab893-ca08-40ef-9a02-583440eb4788\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-f8jvx" Jul 05 13:19:15.408433 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.407662 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svc7w\" (UniqueName: \"kubernetes.io/projected/fab50c14-6777-44c0-a7b7-e3b52ea810ca-kube-api-access-svc7w\") pod \"volume-data-source-validator-5d456fdbcd-8dj7d\" (UID: \"fab50c14-6777-44c0-a7b7-e3b52ea810ca\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8dj7d" Jul 05 13:19:15.408772 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.408704 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2dfp\" (UniqueName: \"kubernetes.io/projected/532a0da4-c9ae-4e33-96e8-381c6715d7e9-kube-api-access-j2dfp\") pod \"insights-operator-5c847dcfc4-7vrt7\" (UID: \"532a0da4-c9ae-4e33-96e8-381c6715d7e9\") " pod="openshift-insights/insights-operator-5c847dcfc4-7vrt7" Jul 05 13:19:15.425323 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.425275 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8dj7d" Jul 05 13:19:15.456064 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.456026 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7tlcm"] Jul 05 13:19:15.458692 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.458658 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-7vrt7" Jul 05 13:19:15.458874 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:19:15.458854 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98016b6f_895f_4fe5_9607_1d2c7ec2f8a3.slice/crio-0a248f7d403fcdca0990f793d28e49310a5dbda22497ab8008008f8d2401d299 WatchSource:0}: Error finding container 0a248f7d403fcdca0990f793d28e49310a5dbda22497ab8008008f8d2401d299: Status 404 returned error can't find the container with id 0a248f7d403fcdca0990f793d28e49310a5dbda22497ab8008008f8d2401d299 Jul 05 13:19:15.499927 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.499302 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bccc5e19-26b8-4f6b-96f2-92684dc3d25a-serving-cert\") pod \"console-operator-68b757fb67-26jcf\" (UID: \"bccc5e19-26b8-4f6b-96f2-92684dc3d25a\") " pod="openshift-console-operator/console-operator-68b757fb67-26jcf" Jul 05 13:19:15.499927 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.499335 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-stats-auth\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:15.499927 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.499355 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-metrics-certs\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:15.499927 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.499373 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bccc5e19-26b8-4f6b-96f2-92684dc3d25a-config\") pod \"console-operator-68b757fb67-26jcf\" (UID: \"bccc5e19-26b8-4f6b-96f2-92684dc3d25a\") " pod="openshift-console-operator/console-operator-68b757fb67-26jcf" Jul 05 13:19:15.499927 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.499390 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j56f\" (UniqueName: \"kubernetes.io/projected/95b0d0a1-07c4-4235-b8d0-5998a2d26473-kube-api-access-7j56f\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:15.499927 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:15.499902 2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 05 13:19:15.500252 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:15.499976 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-metrics-certs podName:95b0d0a1-07c4-4235-b8d0-5998a2d26473 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:15.999952622 +0000 UTC m=+42.340970373 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-metrics-certs") pod "router-default-644877d48f-jctfg" (UID: "95b0d0a1-07c4-4235-b8d0-5998a2d26473") : secret "router-metrics-certs-default" not found Jul 05 13:19:15.500613 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.500562 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdlkd\" (UniqueName: \"kubernetes.io/projected/bccc5e19-26b8-4f6b-96f2-92684dc3d25a-kube-api-access-mdlkd\") pod \"console-operator-68b757fb67-26jcf\" (UID: \"bccc5e19-26b8-4f6b-96f2-92684dc3d25a\") " pod="openshift-console-operator/console-operator-68b757fb67-26jcf" Jul 05 13:19:15.500613 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.500569 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bccc5e19-26b8-4f6b-96f2-92684dc3d25a-config\") pod \"console-operator-68b757fb67-26jcf\" (UID: \"bccc5e19-26b8-4f6b-96f2-92684dc3d25a\") " pod="openshift-console-operator/console-operator-68b757fb67-26jcf" Jul 05 13:19:15.500613 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.500592 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-default-certificate\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:15.500781 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.500639 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bccc5e19-26b8-4f6b-96f2-92684dc3d25a-trusted-ca\") pod \"console-operator-68b757fb67-26jcf\" (UID: \"bccc5e19-26b8-4f6b-96f2-92684dc3d25a\") " pod="openshift-console-operator/console-operator-68b757fb67-26jcf" Jul 05 13:19:15.500781 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.500701 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95b0d0a1-07c4-4235-b8d0-5998a2d26473-service-ca-bundle\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:15.501096 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:15.500961 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/95b0d0a1-07c4-4235-b8d0-5998a2d26473-service-ca-bundle podName:95b0d0a1-07c4-4235-b8d0-5998a2d26473 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:16.000942133 +0000 UTC m=+42.341959884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/95b0d0a1-07c4-4235-b8d0-5998a2d26473-service-ca-bundle") pod "router-default-644877d48f-jctfg" (UID: "95b0d0a1-07c4-4235-b8d0-5998a2d26473") : configmap references non-existent config key: service-ca.crt Jul 05 13:19:15.503501 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.503460 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bccc5e19-26b8-4f6b-96f2-92684dc3d25a-trusted-ca\") pod \"console-operator-68b757fb67-26jcf\" (UID: \"bccc5e19-26b8-4f6b-96f2-92684dc3d25a\") " pod="openshift-console-operator/console-operator-68b757fb67-26jcf" Jul 05 13:19:15.503650 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.503592 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-stats-auth\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:15.505331 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.505270 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-default-certificate\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:15.507526 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.506136 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bccc5e19-26b8-4f6b-96f2-92684dc3d25a-serving-cert\") pod \"console-operator-68b757fb67-26jcf\" (UID: \"bccc5e19-26b8-4f6b-96f2-92684dc3d25a\") " pod="openshift-console-operator/console-operator-68b757fb67-26jcf" Jul 05 13:19:15.512450 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.512386 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j56f\" (UniqueName: \"kubernetes.io/projected/95b0d0a1-07c4-4235-b8d0-5998a2d26473-kube-api-access-7j56f\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:15.512450 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.512432 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdlkd\" (UniqueName: \"kubernetes.io/projected/bccc5e19-26b8-4f6b-96f2-92684dc3d25a-kube-api-access-mdlkd\") pod \"console-operator-68b757fb67-26jcf\" (UID: \"bccc5e19-26b8-4f6b-96f2-92684dc3d25a\") " pod="openshift-console-operator/console-operator-68b757fb67-26jcf" Jul 05 13:19:15.513148 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.513126 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rgmdx" Jul 05 13:19:15.530893 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.529985 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-26jcf" Jul 05 13:19:15.585515 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.584251 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-67485"] Jul 05 13:19:15.591831 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:19:15.591799 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod011559fd_ed92_41fe_acae_079a7650cd20.slice/crio-3c5a9355cddba1ce941458dfedc0eac6faacd0e33c678decdd482a07977c84d1 WatchSource:0}: Error finding container 3c5a9355cddba1ce941458dfedc0eac6faacd0e33c678decdd482a07977c84d1: Status 404 returned error can't find the container with id 3c5a9355cddba1ce941458dfedc0eac6faacd0e33c678decdd482a07977c84d1 Jul 05 13:19:15.622195 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.620079 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8dj7d"] Jul 05 13:19:15.629494 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:19:15.628887 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfab50c14_6777_44c0_a7b7_e3b52ea810ca.slice/crio-224076e8511265db8375f73f6c42f3b6677e8a1be574767440ef5d894c31dbab WatchSource:0}: Error finding container 224076e8511265db8375f73f6c42f3b6677e8a1be574767440ef5d894c31dbab: Status 404 returned error can't find the container with id 224076e8511265db8375f73f6c42f3b6677e8a1be574767440ef5d894c31dbab Jul 05 13:19:15.656998 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.656965 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-7vrt7"] Jul 05 13:19:15.659083 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:19:15.659049 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod532a0da4_c9ae_4e33_96e8_381c6715d7e9.slice/crio-8b6c62249e4421153c2c19a8b98298038fa072b7fc6c575c4074284ef9b9225f WatchSource:0}: Error finding container 8b6c62249e4421153c2c19a8b98298038fa072b7fc6c575c4074284ef9b9225f: Status 404 returned error can't find the container with id 8b6c62249e4421153c2c19a8b98298038fa072b7fc6c575c4074284ef9b9225f Jul 05 13:19:15.713672 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.713619 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rgmdx"] Jul 05 13:19:15.716413 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:19:15.716351 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode21b2ebc_6b56_45d0_ab65_e144505dd4b4.slice/crio-56ff278997f422d979aa62cb43d06d9633ed761e938b7f242195e38d8046c406 WatchSource:0}: Error finding container 56ff278997f422d979aa62cb43d06d9633ed761e938b7f242195e38d8046c406: Status 404 returned error can't find the container with id 56ff278997f422d979aa62cb43d06d9633ed761e938b7f242195e38d8046c406 Jul 05 13:19:15.730433 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.730406 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-26jcf"] Jul 05 13:19:15.733302 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:19:15.733266 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbccc5e19_26b8_4f6b_96f2_92684dc3d25a.slice/crio-0ac51abb9a04fddaece7e859ddf12b7d318237f8b943f7d0cd6bdbaee3d5dc4a WatchSource:0}: Error finding container 0ac51abb9a04fddaece7e859ddf12b7d318237f8b943f7d0cd6bdbaee3d5dc4a: Status 404 returned error can't find the container with id 0ac51abb9a04fddaece7e859ddf12b7d318237f8b943f7d0cd6bdbaee3d5dc4a Jul 05 13:19:15.904928 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.904876 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/352ab893-ca08-40ef-9a02-583440eb4788-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-f8jvx\" (UID: \"352ab893-ca08-40ef-9a02-583440eb4788\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-f8jvx" Jul 05 13:19:15.905133 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:15.905018 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdc1a97f-d106-47bf-92ab-967e6af9ef16-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-xnqg7\" (UID: \"cdc1a97f-d106-47bf-92ab-967e6af9ef16\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-xnqg7" Jul 05 13:19:15.905133 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:15.905020 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 05 13:19:15.905133 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:15.905123 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/352ab893-ca08-40ef-9a02-583440eb4788-cluster-monitoring-operator-tls podName:352ab893-ca08-40ef-9a02-583440eb4788 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:16.905108297 +0000 UTC m=+43.246126049 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/352ab893-ca08-40ef-9a02-583440eb4788-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-f8jvx" (UID: "352ab893-ca08-40ef-9a02-583440eb4788") : secret "cluster-monitoring-operator-tls" not found Jul 05 13:19:15.905133 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:15.905067 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 05 13:19:15.905281 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:15.905183 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdc1a97f-d106-47bf-92ab-967e6af9ef16-samples-operator-tls podName:cdc1a97f-d106-47bf-92ab-967e6af9ef16 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:16.905168145 +0000 UTC m=+43.246185894 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cdc1a97f-d106-47bf-92ab-967e6af9ef16-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-xnqg7" (UID: "cdc1a97f-d106-47bf-92ab-967e6af9ef16") : secret "samples-operator-tls" not found Jul 05 13:19:16.005945 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:16.005857 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95b0d0a1-07c4-4235-b8d0-5998a2d26473-service-ca-bundle\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:16.005945 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:16.005920 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-metrics-certs\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:16.006418 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:16.006042 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/95b0d0a1-07c4-4235-b8d0-5998a2d26473-service-ca-bundle podName:95b0d0a1-07c4-4235-b8d0-5998a2d26473 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:17.006020249 +0000 UTC m=+43.347038000 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/95b0d0a1-07c4-4235-b8d0-5998a2d26473-service-ca-bundle") pod "router-default-644877d48f-jctfg" (UID: "95b0d0a1-07c4-4235-b8d0-5998a2d26473") : configmap references non-existent config key: service-ca.crt Jul 05 13:19:16.006418 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:16.006073 2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 05 13:19:16.006418 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:16.006129 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-metrics-certs podName:95b0d0a1-07c4-4235-b8d0-5998a2d26473 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:17.006116608 +0000 UTC m=+43.347134358 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-metrics-certs") pod "router-default-644877d48f-jctfg" (UID: "95b0d0a1-07c4-4235-b8d0-5998a2d26473") : secret "router-metrics-certs-default" not found Jul 05 13:19:16.416547 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:16.416474 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rgmdx" event={"ID":"e21b2ebc-6b56-45d0-ab65-e144505dd4b4","Type":"ContainerStarted","Data":"56ff278997f422d979aa62cb43d06d9633ed761e938b7f242195e38d8046c406"} Jul 05 13:19:16.418782 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:16.418729 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-26jcf" event={"ID":"bccc5e19-26b8-4f6b-96f2-92684dc3d25a","Type":"ContainerStarted","Data":"0ac51abb9a04fddaece7e859ddf12b7d318237f8b943f7d0cd6bdbaee3d5dc4a"} Jul 05 13:19:16.421719 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:16.421676 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-7vrt7" event={"ID":"532a0da4-c9ae-4e33-96e8-381c6715d7e9","Type":"ContainerStarted","Data":"8b6c62249e4421153c2c19a8b98298038fa072b7fc6c575c4074284ef9b9225f"} Jul 05 13:19:16.423149 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:16.423084 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8dj7d" event={"ID":"fab50c14-6777-44c0-a7b7-e3b52ea810ca","Type":"ContainerStarted","Data":"224076e8511265db8375f73f6c42f3b6677e8a1be574767440ef5d894c31dbab"} Jul 05 13:19:16.424671 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:16.424643 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-67485" event={"ID":"011559fd-ed92-41fe-acae-079a7650cd20","Type":"ContainerStarted","Data":"3c5a9355cddba1ce941458dfedc0eac6faacd0e33c678decdd482a07977c84d1"} Jul 05 13:19:16.427083 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:16.427029 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7tlcm" event={"ID":"98016b6f-895f-4fe5-9607-1d2c7ec2f8a3","Type":"ContainerStarted","Data":"0a248f7d403fcdca0990f793d28e49310a5dbda22497ab8008008f8d2401d299"} Jul 05 13:19:16.916609 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:16.916556 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/352ab893-ca08-40ef-9a02-583440eb4788-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-f8jvx\" (UID: \"352ab893-ca08-40ef-9a02-583440eb4788\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-f8jvx" Jul 05 13:19:16.916803 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:16.916666 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdc1a97f-d106-47bf-92ab-967e6af9ef16-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-xnqg7\" (UID: \"cdc1a97f-d106-47bf-92ab-967e6af9ef16\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-xnqg7" Jul 05 13:19:16.916991 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:16.916943 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 05 13:19:16.917058 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:16.917015 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdc1a97f-d106-47bf-92ab-967e6af9ef16-samples-operator-tls podName:cdc1a97f-d106-47bf-92ab-967e6af9ef16 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:18.91699626 +0000 UTC m=+45.258014017 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cdc1a97f-d106-47bf-92ab-967e6af9ef16-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-xnqg7" (UID: "cdc1a97f-d106-47bf-92ab-967e6af9ef16") : secret "samples-operator-tls" not found Jul 05 13:19:16.917542 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:16.917500 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 05 13:19:16.917623 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:16.917561 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/352ab893-ca08-40ef-9a02-583440eb4788-cluster-monitoring-operator-tls podName:352ab893-ca08-40ef-9a02-583440eb4788 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:18.917544197 +0000 UTC m=+45.258561952 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/352ab893-ca08-40ef-9a02-583440eb4788-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-f8jvx" (UID: "352ab893-ca08-40ef-9a02-583440eb4788") : secret "cluster-monitoring-operator-tls" not found Jul 05 13:19:17.017432 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:17.017388 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95b0d0a1-07c4-4235-b8d0-5998a2d26473-service-ca-bundle\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:17.017985 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:17.017470 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-metrics-certs\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:17.017985 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:17.017690 2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 05 13:19:17.017985 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:17.017774 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-metrics-certs podName:95b0d0a1-07c4-4235-b8d0-5998a2d26473 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:19.017734592 +0000 UTC m=+45.358752348 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-metrics-certs") pod "router-default-644877d48f-jctfg" (UID: "95b0d0a1-07c4-4235-b8d0-5998a2d26473") : secret "router-metrics-certs-default" not found Jul 05 13:19:17.018235 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:17.018201 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/95b0d0a1-07c4-4235-b8d0-5998a2d26473-service-ca-bundle podName:95b0d0a1-07c4-4235-b8d0-5998a2d26473 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:19.018183617 +0000 UTC m=+45.359201371 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/95b0d0a1-07c4-4235-b8d0-5998a2d26473-service-ca-bundle") pod "router-default-644877d48f-jctfg" (UID: "95b0d0a1-07c4-4235-b8d0-5998a2d26473") : configmap references non-existent config key: service-ca.crt Jul 05 13:19:17.320431 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:17.320363 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f14e6cf-c089-437b-90b6-9cae0be31c56-metrics-tls\") pod \"dns-default-mkxp5\" (UID: \"4f14e6cf-c089-437b-90b6-9cae0be31c56\") " pod="openshift-dns/dns-default-mkxp5" Jul 05 13:19:17.320718 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:17.320451 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-registry-tls\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:17.320718 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:17.320533 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:19:17.320718 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:17.320610 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f14e6cf-c089-437b-90b6-9cae0be31c56-metrics-tls podName:4f14e6cf-c089-437b-90b6-9cae0be31c56 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:25.32058937 +0000 UTC m=+51.661607144 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4f14e6cf-c089-437b-90b6-9cae0be31c56-metrics-tls") pod "dns-default-mkxp5" (UID: "4f14e6cf-c089-437b-90b6-9cae0be31c56") : secret "dns-default-metrics-tls" not found Jul 05 13:19:17.320718 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:17.320626 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:19:17.320718 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:17.320641 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7986657c49-hk22t: secret "image-registry-tls" not found Jul 05 13:19:17.320718 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:17.320691 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-registry-tls podName:b1ce72ed-9388-4861-a28b-4f223346551b nodeName:}" failed. No retries permitted until 2026-07-05 13:19:25.320673653 +0000 UTC m=+51.661691424 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-registry-tls") pod "image-registry-7986657c49-hk22t" (UID: "b1ce72ed-9388-4861-a28b-4f223346551b") : secret "image-registry-tls" not found Jul 05 13:19:18.934225 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:18.934186 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/352ab893-ca08-40ef-9a02-583440eb4788-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-f8jvx\" (UID: \"352ab893-ca08-40ef-9a02-583440eb4788\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-f8jvx" Jul 05 13:19:18.934702 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:18.934300 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdc1a97f-d106-47bf-92ab-967e6af9ef16-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-xnqg7\" (UID: \"cdc1a97f-d106-47bf-92ab-967e6af9ef16\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-xnqg7" Jul 05 13:19:18.934702 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:18.934353 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 05 13:19:18.934702 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:18.934419 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/352ab893-ca08-40ef-9a02-583440eb4788-cluster-monitoring-operator-tls podName:352ab893-ca08-40ef-9a02-583440eb4788 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:22.934401901 +0000 UTC m=+49.275419654 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/352ab893-ca08-40ef-9a02-583440eb4788-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-f8jvx" (UID: "352ab893-ca08-40ef-9a02-583440eb4788") : secret "cluster-monitoring-operator-tls" not found Jul 05 13:19:18.934702 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:18.934461 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 05 13:19:18.934702 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:18.934496 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdc1a97f-d106-47bf-92ab-967e6af9ef16-samples-operator-tls podName:cdc1a97f-d106-47bf-92ab-967e6af9ef16 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:22.934485034 +0000 UTC m=+49.275502798 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cdc1a97f-d106-47bf-92ab-967e6af9ef16-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-xnqg7" (UID: "cdc1a97f-d106-47bf-92ab-967e6af9ef16") : secret "samples-operator-tls" not found Jul 05 13:19:19.035039 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:19.034995 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95b0d0a1-07c4-4235-b8d0-5998a2d26473-service-ca-bundle\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:19.035207 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:19.035051 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-metrics-certs\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:19.035207 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:19.035159 2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 05 13:19:19.035207 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:19.035178 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/95b0d0a1-07c4-4235-b8d0-5998a2d26473-service-ca-bundle podName:95b0d0a1-07c4-4235-b8d0-5998a2d26473 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:23.03516028 +0000 UTC m=+49.376178033 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/95b0d0a1-07c4-4235-b8d0-5998a2d26473-service-ca-bundle") pod "router-default-644877d48f-jctfg" (UID: "95b0d0a1-07c4-4235-b8d0-5998a2d26473") : configmap references non-existent config key: service-ca.crt Jul 05 13:19:19.035207 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:19.035205 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-metrics-certs podName:95b0d0a1-07c4-4235-b8d0-5998a2d26473 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:23.035197901 +0000 UTC m=+49.376215651 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-metrics-certs") pod "router-default-644877d48f-jctfg" (UID: "95b0d0a1-07c4-4235-b8d0-5998a2d26473") : secret "router-metrics-certs-default" not found Jul 05 13:19:21.276166 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:21.276132 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-xf8bg"] Jul 05 13:19:21.315190 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:21.315145 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-xf8bg"] Jul 05 13:19:21.315390 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:21.315261 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xf8bg" Jul 05 13:19:21.319269 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:21.319246 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 05 13:19:21.319269 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:21.319261 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 05 13:19:21.319465 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:21.319358 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-6qxxw\"" Jul 05 13:19:21.355452 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:21.355417 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4af1596e-0aa4-4395-839e-49cf9f20ab11-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-xf8bg\" (UID: \"4af1596e-0aa4-4395-839e-49cf9f20ab11\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xf8bg" Jul 05 13:19:21.355648 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:21.355512 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4af1596e-0aa4-4395-839e-49cf9f20ab11-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xf8bg\" (UID: \"4af1596e-0aa4-4395-839e-49cf9f20ab11\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xf8bg" Jul 05 13:19:21.456224 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:21.456173 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4af1596e-0aa4-4395-839e-49cf9f20ab11-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-xf8bg\" (UID: \"4af1596e-0aa4-4395-839e-49cf9f20ab11\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xf8bg" Jul 05 13:19:21.456410 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:21.456264 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4af1596e-0aa4-4395-839e-49cf9f20ab11-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xf8bg\" (UID: \"4af1596e-0aa4-4395-839e-49cf9f20ab11\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xf8bg" Jul 05 13:19:21.456484 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:21.456442 2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 05 13:19:21.456536 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:21.456529 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4af1596e-0aa4-4395-839e-49cf9f20ab11-networking-console-plugin-cert podName:4af1596e-0aa4-4395-839e-49cf9f20ab11 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:21.956504226 +0000 UTC m=+48.297521994 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4af1596e-0aa4-4395-839e-49cf9f20ab11-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xf8bg" (UID: "4af1596e-0aa4-4395-839e-49cf9f20ab11") : secret "networking-console-plugin-cert" not found Jul 05 13:19:21.457018 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:21.456994 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4af1596e-0aa4-4395-839e-49cf9f20ab11-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-xf8bg\" (UID: \"4af1596e-0aa4-4395-839e-49cf9f20ab11\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xf8bg" Jul 05 13:19:21.960300 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:21.960274 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4af1596e-0aa4-4395-839e-49cf9f20ab11-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xf8bg\" (UID: \"4af1596e-0aa4-4395-839e-49cf9f20ab11\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xf8bg" Jul 05 13:19:21.960459 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:21.960439 2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 05 13:19:21.960529 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:21.960519 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4af1596e-0aa4-4395-839e-49cf9f20ab11-networking-console-plugin-cert podName:4af1596e-0aa4-4395-839e-49cf9f20ab11 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:22.960498765 +0000 UTC m=+49.301516522 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4af1596e-0aa4-4395-839e-49cf9f20ab11-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xf8bg" (UID: "4af1596e-0aa4-4395-839e-49cf9f20ab11") : secret "networking-console-plugin-cert" not found Jul 05 13:19:22.448190 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:22.447358 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-26jcf" event={"ID":"bccc5e19-26b8-4f6b-96f2-92684dc3d25a","Type":"ContainerStarted","Data":"b170670f1714b29728eb0871bb1e4e8b2b19a5cfdf89d8c3a04aa83212dbeef2"} Jul 05 13:19:22.448679 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:22.448458 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-26jcf" Jul 05 13:19:22.450444 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:22.449979 2567 patch_prober.go:28] interesting pod/console-operator-68b757fb67-26jcf container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.16:8443/readyz\": dial tcp 10.132.0.16:8443: connect: connection refused" start-of-body= Jul 05 13:19:22.450444 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:22.450032 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-68b757fb67-26jcf" podUID="bccc5e19-26b8-4f6b-96f2-92684dc3d25a" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.16:8443/readyz\": dial tcp 10.132.0.16:8443: connect: connection refused" Jul 05 13:19:22.451462 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:22.451398 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-7vrt7" event={"ID":"532a0da4-c9ae-4e33-96e8-381c6715d7e9","Type":"ContainerStarted","Data":"50b64770e9ee6f268aefdd0991725b391fda1c80d2de1ca153e64839c5f0c742"} Jul 05 13:19:22.453450 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:22.453422 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8dj7d" event={"ID":"fab50c14-6777-44c0-a7b7-e3b52ea810ca","Type":"ContainerStarted","Data":"86e53645b19aacce945f0aaf54998fbaea90790a8050dcd1ee7bced1304c2876"} Jul 05 13:19:22.455411 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:22.455384 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-67485" event={"ID":"011559fd-ed92-41fe-acae-079a7650cd20","Type":"ContainerStarted","Data":"11dafc42f9b84d853c0d8d5191c46c2f20b18b1e12adc91a375b9a1cc49ae467"} Jul 05 13:19:22.457899 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:22.457305 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7tlcm" event={"ID":"98016b6f-895f-4fe5-9607-1d2c7ec2f8a3","Type":"ContainerStarted","Data":"af1cf6a07a4e21e535c92ef60378ea08bab36db8aa7483096361ff9ae1e6eba5"} Jul 05 13:19:22.459163 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:22.459140 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rgmdx" event={"ID":"e21b2ebc-6b56-45d0-ab65-e144505dd4b4","Type":"ContainerStarted","Data":"d6f20dbb7568f5e40c83c127a09adff2ba4021e61a55249a8ecfdad6c50d7bc1"} Jul 05 13:19:22.474073 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:22.474009 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-26jcf" podStartSLOduration=1.292225663 podStartE2EDuration="7.47398707s" totalImagesPullingTime="6.181761407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:22.472096474 +0000 UTC m=+48.813114247" watchObservedRunningTime="2026-07-05 13:19:22.47398707 +0000 UTC m=+48.815004837" Jul 05 13:19:22.507847 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:22.507726 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8dj7d" podStartSLOduration=1.257393166 podStartE2EDuration="7.507702841s" totalImagesPullingTime="6.250309675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:22.506326626 +0000 UTC m=+48.847344395" watchObservedRunningTime="2026-07-05 13:19:22.507702841 +0000 UTC m=+48.848720615" Jul 05 13:19:22.539238 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:22.539174 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rgmdx" podStartSLOduration=1.345539055 podStartE2EDuration="7.539150693s" totalImagesPullingTime="6.193611638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:22.536475765 +0000 UTC m=+48.877493539" watchObservedRunningTime="2026-07-05 13:19:22.539150693 +0000 UTC m=+48.880168469" Jul 05 13:19:22.566572 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:22.566511 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7tlcm" podStartSLOduration=2.115493679 podStartE2EDuration="8.566488208s" totalImagesPullingTime="6.450994529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:22.564728998 +0000 UTC m=+48.905746771" watchObservedRunningTime="2026-07-05 13:19:22.566488208 +0000 UTC m=+48.907505981" Jul 05 13:19:22.592435 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:22.592375 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-67485" podStartSLOduration=0.898969126 podStartE2EDuration="7.592353884s" totalImagesPullingTime="6.693384758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:22.591200976 +0000 UTC m=+48.932218743" watchObservedRunningTime="2026-07-05 13:19:22.592353884 +0000 UTC m=+48.933371658" Jul 05 13:19:22.626451 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:22.626332 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-7vrt7" podStartSLOduration=1.379177905 podStartE2EDuration="7.626310259s" totalImagesPullingTime="6.247132354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:22.625682088 +0000 UTC m=+48.966699861" watchObservedRunningTime="2026-07-05 13:19:22.626310259 +0000 UTC m=+48.967328035" Jul 05 13:19:22.967087 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:22.967036 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/352ab893-ca08-40ef-9a02-583440eb4788-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-f8jvx\" (UID: \"352ab893-ca08-40ef-9a02-583440eb4788\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-f8jvx" Jul 05 13:19:22.967244 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:22.967097 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4af1596e-0aa4-4395-839e-49cf9f20ab11-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xf8bg\" (UID: \"4af1596e-0aa4-4395-839e-49cf9f20ab11\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xf8bg" Jul 05 13:19:22.967244 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:22.967160 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdc1a97f-d106-47bf-92ab-967e6af9ef16-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-xnqg7\" (UID: \"cdc1a97f-d106-47bf-92ab-967e6af9ef16\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-xnqg7" Jul 05 13:19:22.967360 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:22.967347 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 05 13:19:22.967429 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:22.967409 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdc1a97f-d106-47bf-92ab-967e6af9ef16-samples-operator-tls podName:cdc1a97f-d106-47bf-92ab-967e6af9ef16 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:30.967390984 +0000 UTC m=+57.308408743 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cdc1a97f-d106-47bf-92ab-967e6af9ef16-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-xnqg7" (UID: "cdc1a97f-d106-47bf-92ab-967e6af9ef16") : secret "samples-operator-tls" not found Jul 05 13:19:22.967615 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:22.967540 2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 05 13:19:22.967615 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:22.967566 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 05 13:19:22.967615 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:22.967618 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4af1596e-0aa4-4395-839e-49cf9f20ab11-networking-console-plugin-cert podName:4af1596e-0aa4-4395-839e-49cf9f20ab11 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:24.967599622 +0000 UTC m=+51.308617378 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4af1596e-0aa4-4395-839e-49cf9f20ab11-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xf8bg" (UID: "4af1596e-0aa4-4395-839e-49cf9f20ab11") : secret "networking-console-plugin-cert" not found Jul 05 13:19:22.967957 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:22.967638 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/352ab893-ca08-40ef-9a02-583440eb4788-cluster-monitoring-operator-tls podName:352ab893-ca08-40ef-9a02-583440eb4788 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:30.967629084 +0000 UTC m=+57.308646836 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/352ab893-ca08-40ef-9a02-583440eb4788-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-f8jvx" (UID: "352ab893-ca08-40ef-9a02-583440eb4788") : secret "cluster-monitoring-operator-tls" not found Jul 05 13:19:23.067951 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:23.067907 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95b0d0a1-07c4-4235-b8d0-5998a2d26473-service-ca-bundle\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:23.068151 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:23.068007 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-metrics-certs\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:23.068151 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:23.068141 2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 05 13:19:23.068275 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:23.068238 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/95b0d0a1-07c4-4235-b8d0-5998a2d26473-service-ca-bundle podName:95b0d0a1-07c4-4235-b8d0-5998a2d26473 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:31.068218981 +0000 UTC m=+57.409236739 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/95b0d0a1-07c4-4235-b8d0-5998a2d26473-service-ca-bundle") pod "router-default-644877d48f-jctfg" (UID: "95b0d0a1-07c4-4235-b8d0-5998a2d26473") : configmap references non-existent config key: service-ca.crt Jul 05 13:19:23.068275 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:23.068258 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-metrics-certs podName:95b0d0a1-07c4-4235-b8d0-5998a2d26473 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:31.068249159 +0000 UTC m=+57.409266913 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-metrics-certs") pod "router-default-644877d48f-jctfg" (UID: "95b0d0a1-07c4-4235-b8d0-5998a2d26473") : secret "router-metrics-certs-default" not found Jul 05 13:19:23.465516 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:23.465484 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-26jcf_bccc5e19-26b8-4f6b-96f2-92684dc3d25a/console-operator/0.log" Jul 05 13:19:23.466049 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:23.465546 2567 generic.go:356] "Generic (PLEG): container finished" podID="bccc5e19-26b8-4f6b-96f2-92684dc3d25a" containerID="b170670f1714b29728eb0871bb1e4e8b2b19a5cfdf89d8c3a04aa83212dbeef2" exitCode=255 Jul 05 13:19:23.466049 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:23.465643 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-26jcf" event={"ID":"bccc5e19-26b8-4f6b-96f2-92684dc3d25a","Type":"ContainerDied","Data":"b170670f1714b29728eb0871bb1e4e8b2b19a5cfdf89d8c3a04aa83212dbeef2"} Jul 05 13:19:23.466049 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:23.465940 2567 scope.go:117] "RemoveContainer" containerID="b170670f1714b29728eb0871bb1e4e8b2b19a5cfdf89d8c3a04aa83212dbeef2" Jul 05 13:19:23.499573 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:23.499536 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ptp7g"] Jul 05 13:19:23.517501 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:23.517465 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ptp7g" Jul 05 13:19:23.523197 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:23.523139 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 05 13:19:23.523380 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:23.523345 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vjf7b\"" Jul 05 13:19:23.523432 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:23.523417 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 05 13:19:23.533536 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:23.533508 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ptp7g"] Jul 05 13:19:23.573379 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:23.573345 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/524c4576-f63f-40a5-9108-ddc53661f8e8-crio-socket\") pod \"insights-runtime-extractor-ptp7g\" (UID: \"524c4576-f63f-40a5-9108-ddc53661f8e8\") " pod="openshift-insights/insights-runtime-extractor-ptp7g" Jul 05 13:19:23.573561 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:23.573402 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/524c4576-f63f-40a5-9108-ddc53661f8e8-data-volume\") pod \"insights-runtime-extractor-ptp7g\" (UID: \"524c4576-f63f-40a5-9108-ddc53661f8e8\") " pod="openshift-insights/insights-runtime-extractor-ptp7g" Jul 05 13:19:23.573561 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:23.573432 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/524c4576-f63f-40a5-9108-ddc53661f8e8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ptp7g\" (UID: \"524c4576-f63f-40a5-9108-ddc53661f8e8\") " pod="openshift-insights/insights-runtime-extractor-ptp7g" Jul 05 13:19:23.573697 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:23.573660 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8ds5\" (UniqueName: \"kubernetes.io/projected/524c4576-f63f-40a5-9108-ddc53661f8e8-kube-api-access-h8ds5\") pod \"insights-runtime-extractor-ptp7g\" (UID: \"524c4576-f63f-40a5-9108-ddc53661f8e8\") " pod="openshift-insights/insights-runtime-extractor-ptp7g" Jul 05 13:19:23.573852 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:23.573710 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/524c4576-f63f-40a5-9108-ddc53661f8e8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ptp7g\" (UID: \"524c4576-f63f-40a5-9108-ddc53661f8e8\") " pod="openshift-insights/insights-runtime-extractor-ptp7g" Jul 05 13:19:23.675049 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:23.675007 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8ds5\" (UniqueName: \"kubernetes.io/projected/524c4576-f63f-40a5-9108-ddc53661f8e8-kube-api-access-h8ds5\") pod \"insights-runtime-extractor-ptp7g\" (UID: \"524c4576-f63f-40a5-9108-ddc53661f8e8\") " pod="openshift-insights/insights-runtime-extractor-ptp7g" Jul 05 13:19:23.675224 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:23.675065 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/524c4576-f63f-40a5-9108-ddc53661f8e8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ptp7g\" (UID: \"524c4576-f63f-40a5-9108-ddc53661f8e8\") " pod="openshift-insights/insights-runtime-extractor-ptp7g" Jul 05 13:19:23.675224 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:23.675107 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/524c4576-f63f-40a5-9108-ddc53661f8e8-crio-socket\") pod \"insights-runtime-extractor-ptp7g\" (UID: \"524c4576-f63f-40a5-9108-ddc53661f8e8\") " pod="openshift-insights/insights-runtime-extractor-ptp7g" Jul 05 13:19:23.675224 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:23.675148 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/524c4576-f63f-40a5-9108-ddc53661f8e8-data-volume\") pod \"insights-runtime-extractor-ptp7g\" (UID: \"524c4576-f63f-40a5-9108-ddc53661f8e8\") " pod="openshift-insights/insights-runtime-extractor-ptp7g" Jul 05 13:19:23.675224 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:23.675174 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/524c4576-f63f-40a5-9108-ddc53661f8e8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ptp7g\" (UID: \"524c4576-f63f-40a5-9108-ddc53661f8e8\") " pod="openshift-insights/insights-runtime-extractor-ptp7g" Jul 05 13:19:23.675469 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:23.675230 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:19:23.675469 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:23.675282 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 13:19:23.675469 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:23.675303 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/524c4576-f63f-40a5-9108-ddc53661f8e8-insights-runtime-extractor-tls podName:524c4576-f63f-40a5-9108-ddc53661f8e8 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:24.175280656 +0000 UTC m=+50.516298406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/524c4576-f63f-40a5-9108-ddc53661f8e8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ptp7g" (UID: "524c4576-f63f-40a5-9108-ddc53661f8e8") : secret "insights-runtime-extractor-tls" not found Jul 05 13:19:23.675469 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:23.675320 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/524c4576-f63f-40a5-9108-ddc53661f8e8-crio-socket\") pod \"insights-runtime-extractor-ptp7g\" (UID: \"524c4576-f63f-40a5-9108-ddc53661f8e8\") " pod="openshift-insights/insights-runtime-extractor-ptp7g" Jul 05 13:19:23.675469 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:23.675331 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/524c4576-f63f-40a5-9108-ddc53661f8e8-kube-rbac-proxy-cm podName:524c4576-f63f-40a5-9108-ddc53661f8e8 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:24.175316312 +0000 UTC m=+50.516334067 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/524c4576-f63f-40a5-9108-ddc53661f8e8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ptp7g" (UID: "524c4576-f63f-40a5-9108-ddc53661f8e8") : configmap "kube-rbac-proxy" not found Jul 05 13:19:23.675731 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:23.675620 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/524c4576-f63f-40a5-9108-ddc53661f8e8-data-volume\") pod \"insights-runtime-extractor-ptp7g\" (UID: \"524c4576-f63f-40a5-9108-ddc53661f8e8\") " pod="openshift-insights/insights-runtime-extractor-ptp7g" Jul 05 13:19:23.690928 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:23.690893 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8ds5\" (UniqueName: \"kubernetes.io/projected/524c4576-f63f-40a5-9108-ddc53661f8e8-kube-api-access-h8ds5\") pod \"insights-runtime-extractor-ptp7g\" (UID: \"524c4576-f63f-40a5-9108-ddc53661f8e8\") " pod="openshift-insights/insights-runtime-extractor-ptp7g" Jul 05 13:19:24.181260 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:24.181216 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/524c4576-f63f-40a5-9108-ddc53661f8e8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ptp7g\" (UID: \"524c4576-f63f-40a5-9108-ddc53661f8e8\") " pod="openshift-insights/insights-runtime-extractor-ptp7g" Jul 05 13:19:24.181463 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:24.181289 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/524c4576-f63f-40a5-9108-ddc53661f8e8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ptp7g\" (UID: \"524c4576-f63f-40a5-9108-ddc53661f8e8\") " pod="openshift-insights/insights-runtime-extractor-ptp7g" Jul 05 13:19:24.181463 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:24.181399 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:19:24.181463 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:24.181406 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 13:19:24.181569 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:24.181486 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/524c4576-f63f-40a5-9108-ddc53661f8e8-insights-runtime-extractor-tls podName:524c4576-f63f-40a5-9108-ddc53661f8e8 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:25.181466009 +0000 UTC m=+51.522483759 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/524c4576-f63f-40a5-9108-ddc53661f8e8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ptp7g" (UID: "524c4576-f63f-40a5-9108-ddc53661f8e8") : secret "insights-runtime-extractor-tls" not found Jul 05 13:19:24.181569 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:24.181500 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/524c4576-f63f-40a5-9108-ddc53661f8e8-kube-rbac-proxy-cm podName:524c4576-f63f-40a5-9108-ddc53661f8e8 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:25.181493737 +0000 UTC m=+51.522511487 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/524c4576-f63f-40a5-9108-ddc53661f8e8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ptp7g" (UID: "524c4576-f63f-40a5-9108-ddc53661f8e8") : configmap "kube-rbac-proxy" not found Jul 05 13:19:24.470421 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:24.470343 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-26jcf_bccc5e19-26b8-4f6b-96f2-92684dc3d25a/console-operator/1.log" Jul 05 13:19:24.470849 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:24.470728 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-26jcf_bccc5e19-26b8-4f6b-96f2-92684dc3d25a/console-operator/0.log" Jul 05 13:19:24.470849 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:24.470788 2567 generic.go:356] "Generic (PLEG): container finished" podID="bccc5e19-26b8-4f6b-96f2-92684dc3d25a" containerID="94d751b2755a8e9fbc3a0e154cf8e39781bf44e4cc2e297261102316695af3c4" exitCode=255 Jul 05 13:19:24.470930 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:24.470863 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-26jcf" event={"ID":"bccc5e19-26b8-4f6b-96f2-92684dc3d25a","Type":"ContainerDied","Data":"94d751b2755a8e9fbc3a0e154cf8e39781bf44e4cc2e297261102316695af3c4"} Jul 05 13:19:24.481713 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:24.470948 2567 scope.go:117] "RemoveContainer" containerID="b170670f1714b29728eb0871bb1e4e8b2b19a5cfdf89d8c3a04aa83212dbeef2" Jul 05 13:19:24.481906 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:24.481812 2567 scope.go:117] "RemoveContainer" containerID="94d751b2755a8e9fbc3a0e154cf8e39781bf44e4cc2e297261102316695af3c4" Jul 05 13:19:24.482085 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:24.482056 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-26jcf_openshift-console-operator(bccc5e19-26b8-4f6b-96f2-92684dc3d25a)\"" pod="openshift-console-operator/console-operator-68b757fb67-26jcf" podUID="bccc5e19-26b8-4f6b-96f2-92684dc3d25a" Jul 05 13:19:24.864985 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:24.864899 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-78f24_8af4784e-62fd-4a92-ba60-c91f69a2322b/dns-node-resolver/0.log" Jul 05 13:19:24.988041 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:24.988006 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4af1596e-0aa4-4395-839e-49cf9f20ab11-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xf8bg\" (UID: \"4af1596e-0aa4-4395-839e-49cf9f20ab11\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xf8bg" Jul 05 13:19:24.988223 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:24.988180 2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 05 13:19:24.988268 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:24.988250 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4af1596e-0aa4-4395-839e-49cf9f20ab11-networking-console-plugin-cert podName:4af1596e-0aa4-4395-839e-49cf9f20ab11 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:28.98823396 +0000 UTC m=+55.329251710 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4af1596e-0aa4-4395-839e-49cf9f20ab11-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xf8bg" (UID: "4af1596e-0aa4-4395-839e-49cf9f20ab11") : secret "networking-console-plugin-cert" not found Jul 05 13:19:25.189678 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:25.189643 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/524c4576-f63f-40a5-9108-ddc53661f8e8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ptp7g\" (UID: \"524c4576-f63f-40a5-9108-ddc53661f8e8\") " pod="openshift-insights/insights-runtime-extractor-ptp7g" Jul 05 13:19:25.189897 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:25.189701 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/524c4576-f63f-40a5-9108-ddc53661f8e8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ptp7g\" (UID: \"524c4576-f63f-40a5-9108-ddc53661f8e8\") " pod="openshift-insights/insights-runtime-extractor-ptp7g" Jul 05 13:19:25.189897 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:25.189825 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:19:25.189897 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:25.189843 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 13:19:25.189897 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:25.189890 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/524c4576-f63f-40a5-9108-ddc53661f8e8-insights-runtime-extractor-tls podName:524c4576-f63f-40a5-9108-ddc53661f8e8 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:27.189875183 +0000 UTC m=+53.530892934 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/524c4576-f63f-40a5-9108-ddc53661f8e8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ptp7g" (UID: "524c4576-f63f-40a5-9108-ddc53661f8e8") : secret "insights-runtime-extractor-tls" not found Jul 05 13:19:25.190104 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:25.189907 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/524c4576-f63f-40a5-9108-ddc53661f8e8-kube-rbac-proxy-cm podName:524c4576-f63f-40a5-9108-ddc53661f8e8 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:27.189899342 +0000 UTC m=+53.530917093 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/524c4576-f63f-40a5-9108-ddc53661f8e8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ptp7g" (UID: "524c4576-f63f-40a5-9108-ddc53661f8e8") : configmap "kube-rbac-proxy" not found Jul 05 13:19:25.391293 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:25.391256 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f14e6cf-c089-437b-90b6-9cae0be31c56-metrics-tls\") pod \"dns-default-mkxp5\" (UID: \"4f14e6cf-c089-437b-90b6-9cae0be31c56\") " pod="openshift-dns/dns-default-mkxp5" Jul 05 13:19:25.391486 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:25.391334 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-registry-tls\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:25.391486 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:25.391412 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:19:25.391486 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:25.391474 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:19:25.391613 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:25.391493 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7986657c49-hk22t: secret "image-registry-tls" not found Jul 05 13:19:25.391613 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:25.391482 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f14e6cf-c089-437b-90b6-9cae0be31c56-metrics-tls podName:4f14e6cf-c089-437b-90b6-9cae0be31c56 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:41.39146475 +0000 UTC m=+67.732482503 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4f14e6cf-c089-437b-90b6-9cae0be31c56-metrics-tls") pod "dns-default-mkxp5" (UID: "4f14e6cf-c089-437b-90b6-9cae0be31c56") : secret "dns-default-metrics-tls" not found Jul 05 13:19:25.391613 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:25.391547 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-registry-tls podName:b1ce72ed-9388-4861-a28b-4f223346551b nodeName:}" failed. No retries permitted until 2026-07-05 13:19:41.391532785 +0000 UTC m=+67.732550534 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-registry-tls") pod "image-registry-7986657c49-hk22t" (UID: "b1ce72ed-9388-4861-a28b-4f223346551b") : secret "image-registry-tls" not found Jul 05 13:19:25.476079 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:25.475995 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-26jcf_bccc5e19-26b8-4f6b-96f2-92684dc3d25a/console-operator/1.log" Jul 05 13:19:25.476456 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:25.476342 2567 scope.go:117] "RemoveContainer" containerID="94d751b2755a8e9fbc3a0e154cf8e39781bf44e4cc2e297261102316695af3c4" Jul 05 13:19:25.476514 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:25.476494 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-26jcf_openshift-console-operator(bccc5e19-26b8-4f6b-96f2-92684dc3d25a)\"" pod="openshift-console-operator/console-operator-68b757fb67-26jcf" podUID="bccc5e19-26b8-4f6b-96f2-92684dc3d25a" Jul 05 13:19:25.530799 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:25.530755 2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-26jcf" Jul 05 13:19:25.866382 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:25.866300 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8s4z9_84b36d5b-eb96-4252-a0d2-a1f5d8c58a4e/node-ca/0.log" Jul 05 13:19:26.479041 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:26.479014 2567 scope.go:117] "RemoveContainer" containerID="94d751b2755a8e9fbc3a0e154cf8e39781bf44e4cc2e297261102316695af3c4" Jul 05 13:19:26.479473 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:26.479171 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-26jcf_openshift-console-operator(bccc5e19-26b8-4f6b-96f2-92684dc3d25a)\"" pod="openshift-console-operator/console-operator-68b757fb67-26jcf" podUID="bccc5e19-26b8-4f6b-96f2-92684dc3d25a" Jul 05 13:19:26.678150 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:26.678106 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8k5ph_99c92f04-b25b-45b6-aa87-f7085cc847eb/serve-healthcheck-canary/0.log" Jul 05 13:19:27.206276 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:27.206236 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/524c4576-f63f-40a5-9108-ddc53661f8e8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ptp7g\" (UID: \"524c4576-f63f-40a5-9108-ddc53661f8e8\") " pod="openshift-insights/insights-runtime-extractor-ptp7g" Jul 05 13:19:27.206456 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:27.206294 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/524c4576-f63f-40a5-9108-ddc53661f8e8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ptp7g\" (UID: \"524c4576-f63f-40a5-9108-ddc53661f8e8\") " pod="openshift-insights/insights-runtime-extractor-ptp7g" Jul 05 13:19:27.206456 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:27.206412 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:19:27.206532 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:27.206487 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/524c4576-f63f-40a5-9108-ddc53661f8e8-insights-runtime-extractor-tls podName:524c4576-f63f-40a5-9108-ddc53661f8e8 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:31.206469398 +0000 UTC m=+57.547487153 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/524c4576-f63f-40a5-9108-ddc53661f8e8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ptp7g" (UID: "524c4576-f63f-40a5-9108-ddc53661f8e8") : secret "insights-runtime-extractor-tls" not found Jul 05 13:19:27.206842 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:27.206820 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/524c4576-f63f-40a5-9108-ddc53661f8e8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ptp7g\" (UID: \"524c4576-f63f-40a5-9108-ddc53661f8e8\") " pod="openshift-insights/insights-runtime-extractor-ptp7g" Jul 05 13:19:29.022630 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:29.022581 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4af1596e-0aa4-4395-839e-49cf9f20ab11-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xf8bg\" (UID: \"4af1596e-0aa4-4395-839e-49cf9f20ab11\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xf8bg" Jul 05 13:19:29.023043 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:29.022773 2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 05 13:19:29.023043 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:29.022848 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4af1596e-0aa4-4395-839e-49cf9f20ab11-networking-console-plugin-cert podName:4af1596e-0aa4-4395-839e-49cf9f20ab11 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:37.022830636 +0000 UTC m=+63.363848387 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4af1596e-0aa4-4395-839e-49cf9f20ab11-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xf8bg" (UID: "4af1596e-0aa4-4395-839e-49cf9f20ab11") : secret "networking-console-plugin-cert" not found Jul 05 13:19:31.040368 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:31.040323 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdc1a97f-d106-47bf-92ab-967e6af9ef16-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-xnqg7\" (UID: \"cdc1a97f-d106-47bf-92ab-967e6af9ef16\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-xnqg7" Jul 05 13:19:31.040786 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:31.040440 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/352ab893-ca08-40ef-9a02-583440eb4788-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-f8jvx\" (UID: \"352ab893-ca08-40ef-9a02-583440eb4788\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-f8jvx" Jul 05 13:19:31.040786 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:31.040475 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 05 13:19:31.040786 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:31.040552 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 05 13:19:31.040786 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:31.040554 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdc1a97f-d106-47bf-92ab-967e6af9ef16-samples-operator-tls podName:cdc1a97f-d106-47bf-92ab-967e6af9ef16 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:47.040537289 +0000 UTC m=+73.381555039 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cdc1a97f-d106-47bf-92ab-967e6af9ef16-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-xnqg7" (UID: "cdc1a97f-d106-47bf-92ab-967e6af9ef16") : secret "samples-operator-tls" not found Jul 05 13:19:31.040786 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:31.040614 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/352ab893-ca08-40ef-9a02-583440eb4788-cluster-monitoring-operator-tls podName:352ab893-ca08-40ef-9a02-583440eb4788 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:47.04060209 +0000 UTC m=+73.381619840 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/352ab893-ca08-40ef-9a02-583440eb4788-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-f8jvx" (UID: "352ab893-ca08-40ef-9a02-583440eb4788") : secret "cluster-monitoring-operator-tls" not found Jul 05 13:19:31.141234 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:31.141192 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95b0d0a1-07c4-4235-b8d0-5998a2d26473-service-ca-bundle\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:31.141401 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:31.141249 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-metrics-certs\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:31.141401 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:31.141372 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/95b0d0a1-07c4-4235-b8d0-5998a2d26473-service-ca-bundle podName:95b0d0a1-07c4-4235-b8d0-5998a2d26473 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:47.141352367 +0000 UTC m=+73.482370131 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/95b0d0a1-07c4-4235-b8d0-5998a2d26473-service-ca-bundle") pod "router-default-644877d48f-jctfg" (UID: "95b0d0a1-07c4-4235-b8d0-5998a2d26473") : configmap references non-existent config key: service-ca.crt Jul 05 13:19:31.141509 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:31.141408 2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 05 13:19:31.141509 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:31.141471 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-metrics-certs podName:95b0d0a1-07c4-4235-b8d0-5998a2d26473 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:47.141458783 +0000 UTC m=+73.482476533 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-metrics-certs") pod "router-default-644877d48f-jctfg" (UID: "95b0d0a1-07c4-4235-b8d0-5998a2d26473") : secret "router-metrics-certs-default" not found Jul 05 13:19:31.241946 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:31.241900 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/524c4576-f63f-40a5-9108-ddc53661f8e8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ptp7g\" (UID: \"524c4576-f63f-40a5-9108-ddc53661f8e8\") " pod="openshift-insights/insights-runtime-extractor-ptp7g" Jul 05 13:19:31.242221 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:31.242059 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:19:31.242221 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:31.242131 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/524c4576-f63f-40a5-9108-ddc53661f8e8-insights-runtime-extractor-tls podName:524c4576-f63f-40a5-9108-ddc53661f8e8 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:39.242110243 +0000 UTC m=+65.583127993 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/524c4576-f63f-40a5-9108-ddc53661f8e8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ptp7g" (UID: "524c4576-f63f-40a5-9108-ddc53661f8e8") : secret "insights-runtime-extractor-tls" not found Jul 05 13:19:32.449321 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:32.449281 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-26jcf" Jul 05 13:19:32.449707 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:32.449657 2567 scope.go:117] "RemoveContainer" containerID="94d751b2755a8e9fbc3a0e154cf8e39781bf44e4cc2e297261102316695af3c4" Jul 05 13:19:32.449875 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:32.449843 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-26jcf_openshift-console-operator(bccc5e19-26b8-4f6b-96f2-92684dc3d25a)\"" pod="openshift-console-operator/console-operator-68b757fb67-26jcf" podUID="bccc5e19-26b8-4f6b-96f2-92684dc3d25a" Jul 05 13:19:35.382477 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:35.382449 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-swckn" Jul 05 13:19:37.096113 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:37.096067 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4af1596e-0aa4-4395-839e-49cf9f20ab11-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xf8bg\" (UID: \"4af1596e-0aa4-4395-839e-49cf9f20ab11\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xf8bg" Jul 05 13:19:37.096620 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:37.096277 2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 05 13:19:37.096620 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:37.096380 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4af1596e-0aa4-4395-839e-49cf9f20ab11-networking-console-plugin-cert podName:4af1596e-0aa4-4395-839e-49cf9f20ab11 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:53.096356029 +0000 UTC m=+79.437373782 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4af1596e-0aa4-4395-839e-49cf9f20ab11-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xf8bg" (UID: "4af1596e-0aa4-4395-839e-49cf9f20ab11") : secret "networking-console-plugin-cert" not found Jul 05 13:19:39.317327 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:39.317276 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/524c4576-f63f-40a5-9108-ddc53661f8e8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ptp7g\" (UID: \"524c4576-f63f-40a5-9108-ddc53661f8e8\") " pod="openshift-insights/insights-runtime-extractor-ptp7g" Jul 05 13:19:39.319884 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:39.319857 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/524c4576-f63f-40a5-9108-ddc53661f8e8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ptp7g\" (UID: \"524c4576-f63f-40a5-9108-ddc53661f8e8\") " pod="openshift-insights/insights-runtime-extractor-ptp7g" Jul 05 13:19:39.446425 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:39.446391 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vjf7b\"" Jul 05 13:19:39.452856 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:39.452830 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ptp7g" Jul 05 13:19:39.611993 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:39.611898 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ptp7g"] Jul 05 13:19:39.615930 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:19:39.615906 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod524c4576_f63f_40a5_9108_ddc53661f8e8.slice/crio-b57d3f6c4d3be5b2553bb2b058e7c56fe4ec02e72da2dcc4f54b6d1779ec833b WatchSource:0}: Error finding container b57d3f6c4d3be5b2553bb2b058e7c56fe4ec02e72da2dcc4f54b6d1779ec833b: Status 404 returned error can't find the container with id b57d3f6c4d3be5b2553bb2b058e7c56fe4ec02e72da2dcc4f54b6d1779ec833b Jul 05 13:19:39.922681 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:39.922636 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32a4a45d-3b64-476e-9147-09693f809841-original-pull-secret\") pod \"global-pull-secret-syncer-k92vt\" (UID: \"32a4a45d-3b64-476e-9147-09693f809841\") " pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:19:39.922879 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:39.922723 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-metrics-certs\") pod \"network-metrics-daemon-9sgqr\" (UID: \"bcbf773f-ee0a-42bd-8983-a1a3bb066e5d\") " pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:19:39.927942 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:39.927915 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 05 13:19:39.928081 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:39.928014 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 05 13:19:39.935965 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:39.935935 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32a4a45d-3b64-476e-9147-09693f809841-original-pull-secret\") pod \"global-pull-secret-syncer-k92vt\" (UID: \"32a4a45d-3b64-476e-9147-09693f809841\") " pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:19:39.936100 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:39.935948 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcbf773f-ee0a-42bd-8983-a1a3bb066e5d-metrics-certs\") pod \"network-metrics-daemon-9sgqr\" (UID: \"bcbf773f-ee0a-42bd-8983-a1a3bb066e5d\") " pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:19:40.023734 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:40.023691 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6w76\" (UniqueName: \"kubernetes.io/projected/2d509ded-4089-44e1-994f-75d2b4570fce-kube-api-access-x6w76\") pod \"network-check-target-j5rvd\" (UID: \"2d509ded-4089-44e1-994f-75d2b4570fce\") " pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:19:40.026817 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:40.026787 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6w76\" (UniqueName: \"kubernetes.io/projected/2d509ded-4089-44e1-994f-75d2b4570fce-kube-api-access-x6w76\") pod \"network-check-target-j5rvd\" (UID: \"2d509ded-4089-44e1-994f-75d2b4570fce\") " pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:19:40.029414 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:40.029386 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k92vt" Jul 05 13:19:40.039894 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:40.039872 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kn7g2\"" Jul 05 13:19:40.046530 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:40.046438 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sgqr" Jul 05 13:19:40.220668 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:40.220530 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k92vt"] Jul 05 13:19:40.223889 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:19:40.223856 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32a4a45d_3b64_476e_9147_09693f809841.slice/crio-d55b9062db689d6e697f9a7513f7cdba72fcc1948699ed5e6204a3304316746b WatchSource:0}: Error finding container d55b9062db689d6e697f9a7513f7cdba72fcc1948699ed5e6204a3304316746b: Status 404 returned error can't find the container with id d55b9062db689d6e697f9a7513f7cdba72fcc1948699ed5e6204a3304316746b Jul 05 13:19:40.257625 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:40.257588 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9sgqr"] Jul 05 13:19:40.261272 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:19:40.261235 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcbf773f_ee0a_42bd_8983_a1a3bb066e5d.slice/crio-c10386689e4706f7b85fff85c6c98105f57732db6d00cd72ce6304f592be4ad3 WatchSource:0}: Error finding container c10386689e4706f7b85fff85c6c98105f57732db6d00cd72ce6304f592be4ad3: Status 404 returned error can't find the container with id c10386689e4706f7b85fff85c6c98105f57732db6d00cd72ce6304f592be4ad3 Jul 05 13:19:40.326875 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:40.326616 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6n5k8\"" Jul 05 13:19:40.333147 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:40.332854 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:19:40.519864 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:40.519684 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-j5rvd"] Jul 05 13:19:40.523439 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:40.523027 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9sgqr" event={"ID":"bcbf773f-ee0a-42bd-8983-a1a3bb066e5d","Type":"ContainerStarted","Data":"c10386689e4706f7b85fff85c6c98105f57732db6d00cd72ce6304f592be4ad3"} Jul 05 13:19:40.524473 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:40.524421 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k92vt" event={"ID":"32a4a45d-3b64-476e-9147-09693f809841","Type":"ContainerStarted","Data":"d55b9062db689d6e697f9a7513f7cdba72fcc1948699ed5e6204a3304316746b"} Jul 05 13:19:40.527228 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:19:40.527205 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d509ded_4089_44e1_994f_75d2b4570fce.slice/crio-3b28d14545fdbe86f52434b6fea1ac2b967e74d09c0151431d7fd7c0ac608b3d WatchSource:0}: Error finding container 3b28d14545fdbe86f52434b6fea1ac2b967e74d09c0151431d7fd7c0ac608b3d: Status 404 returned error can't find the container with id 3b28d14545fdbe86f52434b6fea1ac2b967e74d09c0151431d7fd7c0ac608b3d Jul 05 13:19:40.530084 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:40.530001 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ptp7g" event={"ID":"524c4576-f63f-40a5-9108-ddc53661f8e8","Type":"ContainerStarted","Data":"b57d3f6c4d3be5b2553bb2b058e7c56fe4ec02e72da2dcc4f54b6d1779ec833b"} Jul 05 13:19:41.437593 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:41.437553 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f14e6cf-c089-437b-90b6-9cae0be31c56-metrics-tls\") pod \"dns-default-mkxp5\" (UID: \"4f14e6cf-c089-437b-90b6-9cae0be31c56\") " pod="openshift-dns/dns-default-mkxp5" Jul 05 13:19:41.438082 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:41.437639 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-registry-tls\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:41.440680 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:41.440648 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-registry-tls\") pod \"image-registry-7986657c49-hk22t\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:41.441572 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:41.441489 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f14e6cf-c089-437b-90b6-9cae0be31c56-metrics-tls\") pod \"dns-default-mkxp5\" (UID: \"4f14e6cf-c089-437b-90b6-9cae0be31c56\") " pod="openshift-dns/dns-default-mkxp5" Jul 05 13:19:41.535398 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:41.535361 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-j5rvd" event={"ID":"2d509ded-4089-44e1-994f-75d2b4570fce","Type":"ContainerStarted","Data":"0ccb6ea0e5a97dd93f9448f4bf6c8dc6625c833aea7144ebd8447ddebe8ea561"} Jul 05 13:19:41.535569 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:41.535416 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-j5rvd" event={"ID":"2d509ded-4089-44e1-994f-75d2b4570fce","Type":"ContainerStarted","Data":"3b28d14545fdbe86f52434b6fea1ac2b967e74d09c0151431d7fd7c0ac608b3d"} Jul 05 13:19:41.535569 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:41.535446 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:19:41.559838 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:41.559787 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-j5rvd" podStartSLOduration=67.559771394 podStartE2EDuration="1m7.559771394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:18:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:41.559454833 +0000 UTC m=+67.900472607" watchObservedRunningTime="2026-07-05 13:19:41.559771394 +0000 UTC m=+67.900789166" Jul 05 13:19:41.655182 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:41.655148 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jcbpr\"" Jul 05 13:19:41.662303 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:41.662267 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:41.673037 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:41.673007 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dww7h\"" Jul 05 13:19:41.680116 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:41.680083 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mkxp5" Jul 05 13:19:45.836846 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:45.833302 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7986657c49-hk22t"] Jul 05 13:19:45.853238 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:45.851115 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mkxp5"] Jul 05 13:19:46.210217 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:46.210182 2567 scope.go:117] "RemoveContainer" containerID="94d751b2755a8e9fbc3a0e154cf8e39781bf44e4cc2e297261102316695af3c4" Jul 05 13:19:46.556529 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:46.556324 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k92vt" event={"ID":"32a4a45d-3b64-476e-9147-09693f809841","Type":"ContainerStarted","Data":"2f7b93e3a8d8d58e9fcb7402da47e05649130334f8bce49eb5fd572329abfafb"} Jul 05 13:19:46.558448 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:46.558419 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ptp7g" event={"ID":"524c4576-f63f-40a5-9108-ddc53661f8e8","Type":"ContainerStarted","Data":"f198ea02e6c410147c50b140ef1ed6be42f2c466ef8f57f95d885f7d48374e37"} Jul 05 13:19:46.558605 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:46.558471 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ptp7g" event={"ID":"524c4576-f63f-40a5-9108-ddc53661f8e8","Type":"ContainerStarted","Data":"e388c8a2921b15f5e26145f343efe15e41a576a50de66b58917696210db3fac9"} Jul 05 13:19:46.560161 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:46.560122 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7986657c49-hk22t" event={"ID":"b1ce72ed-9388-4861-a28b-4f223346551b","Type":"ContainerStarted","Data":"8cd098222ea5fe4c583b2a9de28fe2764dc2a446eb8822f71ca8d1e59216b091"} Jul 05 13:19:46.560300 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:46.560185 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7986657c49-hk22t" event={"ID":"b1ce72ed-9388-4861-a28b-4f223346551b","Type":"ContainerStarted","Data":"544d003b61e5f3e950ab3bf8a67c5ee2798ae6bafb0476858e95e20cc1dabc09"} Jul 05 13:19:46.560372 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:46.560322 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:19:46.562413 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:46.562394 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-26jcf_bccc5e19-26b8-4f6b-96f2-92684dc3d25a/console-operator/1.log" Jul 05 13:19:46.562518 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:46.562475 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-26jcf" event={"ID":"bccc5e19-26b8-4f6b-96f2-92684dc3d25a","Type":"ContainerStarted","Data":"3c76e77c8ca7c1e645b8cdd14479024d9fca73a94fec3d20d9528178847d3bfe"} Jul 05 13:19:46.562803 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:46.562735 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-26jcf" Jul 05 13:19:46.564667 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:46.564641 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9sgqr" event={"ID":"bcbf773f-ee0a-42bd-8983-a1a3bb066e5d","Type":"ContainerStarted","Data":"ae1da77a7249e7ca08cac87814ae9d343a1bc0ad9ef81ee7b2900c4987a19252"} Jul 05 13:19:46.564809 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:46.564682 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9sgqr" event={"ID":"bcbf773f-ee0a-42bd-8983-a1a3bb066e5d","Type":"ContainerStarted","Data":"a1e6914b56d3e12df1ed64bb4020f0241195922fbc907dc7fd400c5c9e635b3e"} Jul 05 13:19:46.565648 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:46.565630 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mkxp5" event={"ID":"4f14e6cf-c089-437b-90b6-9cae0be31c56","Type":"ContainerStarted","Data":"35543f46c142c693e9c88ec4b62c90fd16e1b9dbd643cceeb9761d15d7506759"} Jul 05 13:19:46.576919 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:46.576855 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-k92vt" podStartSLOduration=67.125633304 podStartE2EDuration="1m12.576835715s" totalImagesPullingTime="5.451202411s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:46.57629938 +0000 UTC m=+72.917317153" watchObservedRunningTime="2026-07-05 13:19:46.576835715 +0000 UTC m=+72.917853487" Jul 05 13:19:46.604467 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:46.604386 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7986657c49-hk22t" podStartSLOduration=72.604364076 podStartE2EDuration="1m12.604364076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:18:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:46.600603608 +0000 UTC m=+72.941621379" watchObservedRunningTime="2026-07-05 13:19:46.604364076 +0000 UTC m=+72.945381849" Jul 05 13:19:46.625673 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:46.625608 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9sgqr" podStartSLOduration=67.240591415 podStartE2EDuration="1m12.625590274s" totalImagesPullingTime="5.384998859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:46.623584999 +0000 UTC m=+72.964602770" watchObservedRunningTime="2026-07-05 13:19:46.625590274 +0000 UTC m=+72.966608050" Jul 05 13:19:47.005034 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:47.004824 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-26jcf" Jul 05 13:19:47.091906 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:47.091664 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/352ab893-ca08-40ef-9a02-583440eb4788-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-f8jvx\" (UID: \"352ab893-ca08-40ef-9a02-583440eb4788\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-f8jvx" Jul 05 13:19:47.091906 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:47.091786 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdc1a97f-d106-47bf-92ab-967e6af9ef16-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-xnqg7\" (UID: \"cdc1a97f-d106-47bf-92ab-967e6af9ef16\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-xnqg7" Jul 05 13:19:47.103093 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:47.101948 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdc1a97f-d106-47bf-92ab-967e6af9ef16-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-xnqg7\" (UID: \"cdc1a97f-d106-47bf-92ab-967e6af9ef16\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-xnqg7" Jul 05 13:19:47.114609 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:47.113397 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/352ab893-ca08-40ef-9a02-583440eb4788-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-f8jvx\" (UID: \"352ab893-ca08-40ef-9a02-583440eb4788\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-f8jvx" Jul 05 13:19:47.198394 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:47.194575 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95b0d0a1-07c4-4235-b8d0-5998a2d26473-service-ca-bundle\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:47.198394 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:47.194651 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-metrics-certs\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:47.198394 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:47.195481 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95b0d0a1-07c4-4235-b8d0-5998a2d26473-service-ca-bundle\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:47.198610 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:47.198520 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95b0d0a1-07c4-4235-b8d0-5998a2d26473-metrics-certs\") pod \"router-default-644877d48f-jctfg\" (UID: \"95b0d0a1-07c4-4235-b8d0-5998a2d26473\") " pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:47.247911 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:47.247654 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-7wcx9\"" Jul 05 13:19:47.254328 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:47.254299 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-xnqg7" Jul 05 13:19:47.280774 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:47.280550 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-kskdk\"" Jul 05 13:19:47.284629 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:47.284596 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-f8jvx" Jul 05 13:19:47.354898 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:47.354667 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-hsq9p\"" Jul 05 13:19:47.361523 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:47.361487 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:47.676229 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:47.673984 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-xnqg7"] Jul 05 13:19:47.711912 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:47.711847 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-644877d48f-jctfg"] Jul 05 13:19:47.723971 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:47.723790 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-f8jvx"] Jul 05 13:19:48.585324 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:48.585239 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-xnqg7" event={"ID":"cdc1a97f-d106-47bf-92ab-967e6af9ef16","Type":"ContainerStarted","Data":"54470c6e7a606ff1b29827a8547fe6a19144983365b2e0eb4e73de461647d66a"} Jul 05 13:19:48.587974 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:48.587940 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mkxp5" event={"ID":"4f14e6cf-c089-437b-90b6-9cae0be31c56","Type":"ContainerStarted","Data":"f304557676a7eb4da6c17c3d1ae05d339d8696e67e9d01068457af9669b93ba0"} Jul 05 13:19:48.588122 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:48.588003 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mkxp5" event={"ID":"4f14e6cf-c089-437b-90b6-9cae0be31c56","Type":"ContainerStarted","Data":"2e5b233e407454d49b6c71c9f285ad7e466b3a909612a6ba8e8daa10c435e5fa"} Jul 05 13:19:48.588122 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:48.588113 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mkxp5" Jul 05 13:19:48.590551 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:48.590522 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-f8jvx" event={"ID":"352ab893-ca08-40ef-9a02-583440eb4788","Type":"ContainerStarted","Data":"6e2fa734737450aa29fd5c610cd92e7eda420350e69dc2f3b7eedfddeb485193"} Jul 05 13:19:48.593139 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:48.592675 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-644877d48f-jctfg" event={"ID":"95b0d0a1-07c4-4235-b8d0-5998a2d26473","Type":"ContainerStarted","Data":"94672d2e4df17d27c6f3b17c9dc70c100bec2d61690355d10833b85789920ec9"} Jul 05 13:19:48.593139 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:48.592721 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-644877d48f-jctfg" event={"ID":"95b0d0a1-07c4-4235-b8d0-5998a2d26473","Type":"ContainerStarted","Data":"88ff08b81e5d88358d80ec284b22efb4eb77f20b5583bc14804a6a60e1c7e9ca"} Jul 05 13:19:48.658809 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:48.658712 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mkxp5" podStartSLOduration=37.980731905 podStartE2EDuration="39.658690853s" totalImagesPullingTime="1.677958948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:48.617963684 +0000 UTC m=+74.958981457" watchObservedRunningTime="2026-07-05 13:19:48.658690853 +0000 UTC m=+74.999708624" Jul 05 13:19:48.660285 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:48.660237 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-644877d48f-jctfg" podStartSLOduration=33.66022373 podStartE2EDuration="33.66022373s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:48.657289942 +0000 UTC m=+74.998307714" watchObservedRunningTime="2026-07-05 13:19:48.66022373 +0000 UTC m=+75.001241485" Jul 05 13:19:49.362512 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:49.362466 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:49.364814 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:49.364788 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:49.598391 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:49.598352 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:49.599524 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:49.599501 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-644877d48f-jctfg" Jul 05 13:19:51.609308 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:51.609267 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-f8jvx" event={"ID":"352ab893-ca08-40ef-9a02-583440eb4788","Type":"ContainerStarted","Data":"8d6c9c83d3b70bfd32ae8a02145428406b00dc34d28e5bbc30a92cc151869789"} Jul 05 13:19:51.611427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:51.611397 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ptp7g" event={"ID":"524c4576-f63f-40a5-9108-ddc53661f8e8","Type":"ContainerStarted","Data":"82080d6b520741100f57a4c7cbbbc7910e475576e1a6a6a9b1018439bb93e1ee"} Jul 05 13:19:51.613107 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:51.613077 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-xnqg7" event={"ID":"cdc1a97f-d106-47bf-92ab-967e6af9ef16","Type":"ContainerStarted","Data":"5e5e43a3c4fcb64708e9ed09c38be5a128dcb3f850dc22ac26f1d1cd6f1cda63"} Jul 05 13:19:51.613240 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:51.613129 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-xnqg7" event={"ID":"cdc1a97f-d106-47bf-92ab-967e6af9ef16","Type":"ContainerStarted","Data":"a18ddfec9db20ae6dd9aea74781d8d7660bead204d89edb3b4ec3acd4a229b2e"} Jul 05 13:19:51.631245 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:51.631189 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-f8jvx" podStartSLOduration=33.556128308 podStartE2EDuration="36.631170995s" totalImagesPullingTime="3.075042687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:51.630486501 +0000 UTC m=+77.971504272" watchObservedRunningTime="2026-07-05 13:19:51.631170995 +0000 UTC m=+77.972188766" Jul 05 13:19:51.651792 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:51.651710 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-xnqg7" podStartSLOduration=33.583917412 podStartE2EDuration="36.651693375s" totalImagesPullingTime="3.067775963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:51.651476972 +0000 UTC m=+77.992494746" watchObservedRunningTime="2026-07-05 13:19:51.651693375 +0000 UTC m=+77.992711146" Jul 05 13:19:51.685295 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:51.685237 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ptp7g" podStartSLOduration=17.683930015 podStartE2EDuration="28.685221086s" totalImagesPullingTime="11.001291071s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:19:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:51.683993268 +0000 UTC m=+78.025011040" watchObservedRunningTime="2026-07-05 13:19:51.685221086 +0000 UTC m=+78.026238885" Jul 05 13:19:53.149139 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:53.149041 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4af1596e-0aa4-4395-839e-49cf9f20ab11-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xf8bg\" (UID: \"4af1596e-0aa4-4395-839e-49cf9f20ab11\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xf8bg" Jul 05 13:19:53.151564 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:53.151531 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4af1596e-0aa4-4395-839e-49cf9f20ab11-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xf8bg\" (UID: \"4af1596e-0aa4-4395-839e-49cf9f20ab11\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xf8bg" Jul 05 13:19:53.429874 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:53.429845 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-6qxxw\"" Jul 05 13:19:53.436310 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:53.436282 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xf8bg" Jul 05 13:19:53.594593 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:53.594562 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-xf8bg"] Jul 05 13:19:53.597914 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:19:53.597881 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4af1596e_0aa4_4395_839e_49cf9f20ab11.slice/crio-2495cc29093f9fa79aa0e11f6908730fd4cecd62512a46fa8f8579d21e586c68 WatchSource:0}: Error finding container 2495cc29093f9fa79aa0e11f6908730fd4cecd62512a46fa8f8579d21e586c68: Status 404 returned error can't find the container with id 2495cc29093f9fa79aa0e11f6908730fd4cecd62512a46fa8f8579d21e586c68 Jul 05 13:19:53.621425 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:53.621391 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xf8bg" event={"ID":"4af1596e-0aa4-4395-839e-49cf9f20ab11","Type":"ContainerStarted","Data":"2495cc29093f9fa79aa0e11f6908730fd4cecd62512a46fa8f8579d21e586c68"} Jul 05 13:19:55.630367 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:55.630323 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xf8bg" event={"ID":"4af1596e-0aa4-4395-839e-49cf9f20ab11","Type":"ContainerStarted","Data":"07e210c5dc2386a8b2b415821a93c9bbe389bd2a33b347d4a6fbc6685ce3187a"} Jul 05 13:19:55.650259 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:55.650207 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xf8bg" podStartSLOduration=33.26595901 podStartE2EDuration="34.650185228s" totalImagesPullingTime="1.384226218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:55.649333719 +0000 UTC m=+81.990351494" watchObservedRunningTime="2026-07-05 13:19:55.650185228 +0000 UTC m=+81.991203000" Jul 05 13:19:56.491897 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:56.491862 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-97hz7"] Jul 05 13:19:56.494492 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:56.494473 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-97hz7" Jul 05 13:19:56.497805 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:56.497780 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 05 13:19:56.497936 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:56.497815 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 05 13:19:56.498486 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:56.498470 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-46bkn\"" Jul 05 13:19:56.500154 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:56.500133 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 05 13:19:56.507045 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:56.507018 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-97hz7"] Jul 05 13:19:56.574627 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:56.574591 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-97hz7\" (UID: \"e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-97hz7" Jul 05 13:19:56.574627 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:56.574628 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-97hz7\" (UID: \"e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-97hz7" Jul 05 13:19:56.574905 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:56.574717 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xpnh\" (UniqueName: \"kubernetes.io/projected/e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d-kube-api-access-4xpnh\") pod \"prometheus-operator-567f7b8b7c-97hz7\" (UID: \"e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-97hz7" Jul 05 13:19:56.574905 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:56.574826 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-97hz7\" (UID: \"e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-97hz7" Jul 05 13:19:56.675518 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:56.675469 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xpnh\" (UniqueName: \"kubernetes.io/projected/e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d-kube-api-access-4xpnh\") pod \"prometheus-operator-567f7b8b7c-97hz7\" (UID: \"e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-97hz7" Jul 05 13:19:56.675518 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:56.675516 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-97hz7\" (UID: \"e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-97hz7" Jul 05 13:19:56.676022 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:56.675621 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-97hz7\" (UID: \"e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-97hz7" Jul 05 13:19:56.676022 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:56.675643 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-97hz7\" (UID: \"e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-97hz7" Jul 05 13:19:56.676022 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:56.675764 2567 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 05 13:19:56.676022 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:19:56.675827 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d-prometheus-operator-tls podName:e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d nodeName:}" failed. No retries permitted until 2026-07-05 13:19:57.175812456 +0000 UTC m=+83.516830205 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-97hz7" (UID: "e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d") : secret "prometheus-operator-tls" not found Jul 05 13:19:56.676348 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:56.676329 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-97hz7\" (UID: \"e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-97hz7" Jul 05 13:19:56.678071 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:56.678050 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-97hz7\" (UID: \"e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-97hz7" Jul 05 13:19:56.687445 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:56.687411 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xpnh\" (UniqueName: \"kubernetes.io/projected/e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d-kube-api-access-4xpnh\") pod \"prometheus-operator-567f7b8b7c-97hz7\" (UID: \"e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-97hz7" Jul 05 13:19:57.179858 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:57.179814 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-97hz7\" (UID: \"e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-97hz7" Jul 05 13:19:57.182342 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:57.182311 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-97hz7\" (UID: \"e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-97hz7" Jul 05 13:19:57.404405 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:57.404364 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-97hz7" Jul 05 13:19:57.550423 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:57.550249 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-97hz7"] Jul 05 13:19:57.552942 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:19:57.552921 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9fa9507_2afd_44c3_980a_6d3b9fbbfd5d.slice/crio-3d2f7652f739b22224165c7504454429d8afa5b669203b684baa62b3a1fbe8ce WatchSource:0}: Error finding container 3d2f7652f739b22224165c7504454429d8afa5b669203b684baa62b3a1fbe8ce: Status 404 returned error can't find the container with id 3d2f7652f739b22224165c7504454429d8afa5b669203b684baa62b3a1fbe8ce Jul 05 13:19:57.639082 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:57.639042 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-97hz7" event={"ID":"e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d","Type":"ContainerStarted","Data":"3d2f7652f739b22224165c7504454429d8afa5b669203b684baa62b3a1fbe8ce"} Jul 05 13:19:58.600197 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:58.600168 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mkxp5" Jul 05 13:19:59.649913 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:59.649872 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-97hz7" event={"ID":"e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d","Type":"ContainerStarted","Data":"d131f481696616843a849e580750c318188dfba4a37f88acf364509d18e3e2c1"} Jul 05 13:19:59.650274 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:59.649938 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-97hz7" event={"ID":"e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d","Type":"ContainerStarted","Data":"dd5ed38596214998807ab628754736be34bbe89f720e3b1f646a88aa4caf1900"} Jul 05 13:19:59.673994 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:19:59.673940 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-97hz7" podStartSLOduration=2.525071691 podStartE2EDuration="3.673921522s" totalImagesPullingTime="1.148849831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:59.672505683 +0000 UTC m=+86.013523457" watchObservedRunningTime="2026-07-05 13:19:59.673921522 +0000 UTC m=+86.014939294" Jul 05 13:20:02.329692 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.329651 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mkz5g"] Jul 05 13:20:02.333179 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.333157 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.336082 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.336059 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 05 13:20:02.336232 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.336065 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vrcb7\"" Jul 05 13:20:02.336445 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.336430 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 05 13:20:02.337183 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.337165 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 05 13:20:02.430952 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.430908 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b7a4d23-a0e2-45c8-969e-ef29063451b8-metrics-client-ca\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.430952 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.430943 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6b7a4d23-a0e2-45c8-969e-ef29063451b8-node-exporter-accelerators-collector-config\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.431156 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.430966 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6b7a4d23-a0e2-45c8-969e-ef29063451b8-node-exporter-tls\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.431156 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.431047 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b7a4d23-a0e2-45c8-969e-ef29063451b8-sys\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.431156 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.431096 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6b7a4d23-a0e2-45c8-969e-ef29063451b8-root\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.431156 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.431116 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6b7a4d23-a0e2-45c8-969e-ef29063451b8-node-exporter-wtmp\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.431281 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.431163 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6l8v\" (UniqueName: \"kubernetes.io/projected/6b7a4d23-a0e2-45c8-969e-ef29063451b8-kube-api-access-t6l8v\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.431281 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.431206 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6b7a4d23-a0e2-45c8-969e-ef29063451b8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.431281 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.431245 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6b7a4d23-a0e2-45c8-969e-ef29063451b8-node-exporter-textfile\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.532014 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.531975 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6l8v\" (UniqueName: \"kubernetes.io/projected/6b7a4d23-a0e2-45c8-969e-ef29063451b8-kube-api-access-t6l8v\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.532014 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.532019 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6b7a4d23-a0e2-45c8-969e-ef29063451b8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.532268 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.532055 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6b7a4d23-a0e2-45c8-969e-ef29063451b8-node-exporter-textfile\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.532268 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.532097 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b7a4d23-a0e2-45c8-969e-ef29063451b8-metrics-client-ca\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.532268 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.532120 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6b7a4d23-a0e2-45c8-969e-ef29063451b8-node-exporter-accelerators-collector-config\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.532268 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.532151 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6b7a4d23-a0e2-45c8-969e-ef29063451b8-node-exporter-tls\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.532268 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.532187 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b7a4d23-a0e2-45c8-969e-ef29063451b8-sys\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.532268 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.532225 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6b7a4d23-a0e2-45c8-969e-ef29063451b8-root\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.532268 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.532252 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6b7a4d23-a0e2-45c8-969e-ef29063451b8-node-exporter-wtmp\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.532550 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.532361 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b7a4d23-a0e2-45c8-969e-ef29063451b8-sys\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.532550 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.532409 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6b7a4d23-a0e2-45c8-969e-ef29063451b8-root\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.532550 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.532412 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6b7a4d23-a0e2-45c8-969e-ef29063451b8-node-exporter-wtmp\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.532550 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.532495 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6b7a4d23-a0e2-45c8-969e-ef29063451b8-node-exporter-textfile\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.532800 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.532775 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b7a4d23-a0e2-45c8-969e-ef29063451b8-metrics-client-ca\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.532863 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.532787 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6b7a4d23-a0e2-45c8-969e-ef29063451b8-node-exporter-accelerators-collector-config\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.534724 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.534700 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6b7a4d23-a0e2-45c8-969e-ef29063451b8-node-exporter-tls\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.534841 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.534819 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6b7a4d23-a0e2-45c8-969e-ef29063451b8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.546267 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.546236 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6l8v\" (UniqueName: \"kubernetes.io/projected/6b7a4d23-a0e2-45c8-969e-ef29063451b8-kube-api-access-t6l8v\") pod \"node-exporter-mkz5g\" (UID: \"6b7a4d23-a0e2-45c8-969e-ef29063451b8\") " pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.642268 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.642177 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mkz5g" Jul 05 13:20:02.653053 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:20:02.653023 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b7a4d23_a0e2_45c8_969e_ef29063451b8.slice/crio-656141017feaf5ceb23f084b46773c3431c2d9c02563d1869fe1fce71ce408e7 WatchSource:0}: Error finding container 656141017feaf5ceb23f084b46773c3431c2d9c02563d1869fe1fce71ce408e7: Status 404 returned error can't find the container with id 656141017feaf5ceb23f084b46773c3431c2d9c02563d1869fe1fce71ce408e7 Jul 05 13:20:02.662665 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:02.662632 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mkz5g" event={"ID":"6b7a4d23-a0e2-45c8-969e-ef29063451b8","Type":"ContainerStarted","Data":"656141017feaf5ceb23f084b46773c3431c2d9c02563d1869fe1fce71ce408e7"} Jul 05 13:20:03.313493 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.313454 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 05 13:20:03.316927 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.316897 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.322193 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.321991 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 05 13:20:03.322193 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.321996 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 05 13:20:03.322426 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.322384 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 05 13:20:03.322707 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.322681 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 05 13:20:03.322707 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.322699 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 05 13:20:03.322952 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.322807 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 05 13:20:03.322952 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.322932 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5ttvt\"" Jul 05 13:20:03.323088 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.323074 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 05 13:20:03.323524 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.323504 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 05 13:20:03.332715 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.332687 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 05 13:20:03.361963 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.361927 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 05 13:20:03.441158 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.441121 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.441158 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.441163 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.441393 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.441183 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-web-config\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.441393 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.441247 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c19f332e-4d92-4078-942b-dd95ef711981-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.441393 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.441290 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c19f332e-4d92-4078-942b-dd95ef711981-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.441393 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.441306 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.441393 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.441327 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c19f332e-4d92-4078-942b-dd95ef711981-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.441622 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.441412 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c19f332e-4d92-4078-942b-dd95ef711981-config-out\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.441622 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.441452 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-config-volume\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.441622 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.441475 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.441622 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.441498 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.441622 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.441514 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9pr9\" (UniqueName: \"kubernetes.io/projected/c19f332e-4d92-4078-942b-dd95ef711981-kube-api-access-f9pr9\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.441622 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.441537 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c19f332e-4d92-4078-942b-dd95ef711981-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.542717 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.542680 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c19f332e-4d92-4078-942b-dd95ef711981-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.542932 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.542729 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c19f332e-4d92-4078-942b-dd95ef711981-config-out\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.542932 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.542787 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-config-volume\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.542932 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.542808 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.542932 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.542835 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.542932 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.542858 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9pr9\" (UniqueName: \"kubernetes.io/projected/c19f332e-4d92-4078-942b-dd95ef711981-kube-api-access-f9pr9\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.542932 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.542881 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c19f332e-4d92-4078-942b-dd95ef711981-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.542932 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.542922 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.543148 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.542938 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.543148 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.542956 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-web-config\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.543148 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.542975 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c19f332e-4d92-4078-942b-dd95ef711981-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.543148 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.543016 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c19f332e-4d92-4078-942b-dd95ef711981-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.543148 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.543033 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.543530 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.543424 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c19f332e-4d92-4078-942b-dd95ef711981-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.544690 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.544357 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c19f332e-4d92-4078-942b-dd95ef711981-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.545139 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.545116 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c19f332e-4d92-4078-942b-dd95ef711981-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.546835 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.546815 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.548234 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.548190 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.548392 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.548365 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.548649 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.548630 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-web-config\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.548936 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.548904 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.549060 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.549040 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-config-volume\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.549546 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.549525 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c19f332e-4d92-4078-942b-dd95ef711981-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.549818 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.549794 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.549914 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.549825 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c19f332e-4d92-4078-942b-dd95ef711981-config-out\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.560050 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.560022 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9pr9\" (UniqueName: \"kubernetes.io/projected/c19f332e-4d92-4078-942b-dd95ef711981-kube-api-access-f9pr9\") pod \"alertmanager-main-0\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.630945 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.630873 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:03.843463 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:03.843432 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 05 13:20:03.880025 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:20:03.879976 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc19f332e_4d92_4078_942b_dd95ef711981.slice/crio-9bae6fc4a10f4bf237d890d9cb09aaf0f030a5bf9f87d706cc7a4afa50038a65 WatchSource:0}: Error finding container 9bae6fc4a10f4bf237d890d9cb09aaf0f030a5bf9f87d706cc7a4afa50038a65: Status 404 returned error can't find the container with id 9bae6fc4a10f4bf237d890d9cb09aaf0f030a5bf9f87d706cc7a4afa50038a65 Jul 05 13:20:04.672382 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:04.672338 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c19f332e-4d92-4078-942b-dd95ef711981","Type":"ContainerStarted","Data":"9bae6fc4a10f4bf237d890d9cb09aaf0f030a5bf9f87d706cc7a4afa50038a65"} Jul 05 13:20:04.674383 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:04.674352 2567 generic.go:356] "Generic (PLEG): container finished" podID="6b7a4d23-a0e2-45c8-969e-ef29063451b8" containerID="c628fd6e13af6e9c0a8bb6ee1a926a313d8254259c22ad4b66a308188ba5ef78" exitCode=0 Jul 05 13:20:04.674497 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:04.674405 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mkz5g" event={"ID":"6b7a4d23-a0e2-45c8-969e-ef29063451b8","Type":"ContainerDied","Data":"c628fd6e13af6e9c0a8bb6ee1a926a313d8254259c22ad4b66a308188ba5ef78"} Jul 05 13:20:05.521170 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.521131 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-78d9c755bc-rnqlg"] Jul 05 13:20:05.523873 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.523856 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:05.537932 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.537899 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 05 13:20:05.547507 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:20:05.547470 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"thanos-querier-kube-rbac-proxy-web\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" type="*v1.Secret" Jul 05 13:20:05.547713 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:20:05.547574 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"thanos-querier-78d9c755bc-rnqlg\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="e998c650-79fc-453d-8f79-71ae0703303e" pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:05.547713 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:20:05.547602 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"thanos-querier-dockercfg-t4277\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-t4277\"" type="*v1.Secret" Jul 05 13:20:05.547713 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:20:05.547602 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"thanos-querier-grpc-tls-ejih2go9u3e5m\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-ejih2go9u3e5m\"" type="*v1.Secret" Jul 05 13:20:05.547896 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.547773 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 05 13:20:05.547896 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:20:05.547770 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"thanos-querier-kube-rbac-proxy-rules\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" type="*v1.Secret" Jul 05 13:20:05.547896 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.547774 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 05 13:20:05.575226 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.575197 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-78d9c755bc-rnqlg"] Jul 05 13:20:05.661955 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.661913 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e998c650-79fc-453d-8f79-71ae0703303e-metrics-client-ca\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:05.662133 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.661967 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e998c650-79fc-453d-8f79-71ae0703303e-secret-thanos-querier-tls\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:05.662133 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.662046 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e998c650-79fc-453d-8f79-71ae0703303e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:05.662133 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.662112 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e998c650-79fc-453d-8f79-71ae0703303e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:05.662244 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.662154 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e998c650-79fc-453d-8f79-71ae0703303e-secret-grpc-tls\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:05.662244 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.662180 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9hr2\" (UniqueName: \"kubernetes.io/projected/e998c650-79fc-453d-8f79-71ae0703303e-kube-api-access-s9hr2\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:05.662244 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.662210 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e998c650-79fc-453d-8f79-71ae0703303e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:05.662337 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.662245 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e998c650-79fc-453d-8f79-71ae0703303e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:05.680309 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.680253 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mkz5g" event={"ID":"6b7a4d23-a0e2-45c8-969e-ef29063451b8","Type":"ContainerStarted","Data":"912ff5f2c38f152c634fbec7e08aeb0bff9ebf3f20fa5e6ccf1bfc3ff483e36f"} Jul 05 13:20:05.680770 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.680335 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mkz5g" event={"ID":"6b7a4d23-a0e2-45c8-969e-ef29063451b8","Type":"ContainerStarted","Data":"0a521c39c7a3800920b2d8b30d8199cce066d1ff2fd2519d5c24867d31c9d5d8"} Jul 05 13:20:05.681866 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.681840 2567 generic.go:356] "Generic (PLEG): container finished" podID="c19f332e-4d92-4078-942b-dd95ef711981" containerID="f0de24ccaa56ce8d29e867889225d77e4fa9a20704fefd6f87e2bd6dfd659c38" exitCode=0 Jul 05 13:20:05.681966 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.681933 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c19f332e-4d92-4078-942b-dd95ef711981","Type":"ContainerDied","Data":"f0de24ccaa56ce8d29e867889225d77e4fa9a20704fefd6f87e2bd6dfd659c38"} Jul 05 13:20:05.763457 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.763400 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e998c650-79fc-453d-8f79-71ae0703303e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:05.763660 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.763482 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e998c650-79fc-453d-8f79-71ae0703303e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:05.763660 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.763523 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e998c650-79fc-453d-8f79-71ae0703303e-secret-grpc-tls\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:05.763660 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.763565 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9hr2\" (UniqueName: \"kubernetes.io/projected/e998c650-79fc-453d-8f79-71ae0703303e-kube-api-access-s9hr2\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:05.763660 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.763609 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e998c650-79fc-453d-8f79-71ae0703303e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:05.763660 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.763644 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e998c650-79fc-453d-8f79-71ae0703303e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:05.763957 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.763690 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e998c650-79fc-453d-8f79-71ae0703303e-metrics-client-ca\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:05.763957 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.763761 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e998c650-79fc-453d-8f79-71ae0703303e-secret-thanos-querier-tls\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:05.769015 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.765555 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e998c650-79fc-453d-8f79-71ae0703303e-metrics-client-ca\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:05.769015 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.767040 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e998c650-79fc-453d-8f79-71ae0703303e-secret-thanos-querier-tls\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:05.769015 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.767514 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e998c650-79fc-453d-8f79-71ae0703303e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:05.770691 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.770667 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e998c650-79fc-453d-8f79-71ae0703303e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:05.784052 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.783953 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mkz5g" podStartSLOduration=2.695257058 podStartE2EDuration="3.783937746s" totalImagesPullingTime="1.007095717s" totalInitContainerRuntime="81.584971ms" isStatefulPod=true podCreationTimestamp="2026-07-05 13:20:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:05.783026785 +0000 UTC m=+92.124044569" watchObservedRunningTime="2026-07-05 13:20:05.783937746 +0000 UTC m=+92.124955518" Jul 05 13:20:05.786336 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:05.786314 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9hr2\" (UniqueName: \"kubernetes.io/projected/e998c650-79fc-453d-8f79-71ae0703303e-kube-api-access-s9hr2\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:06.447667 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:06.447628 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 05 13:20:06.457081 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:06.457050 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e998c650-79fc-453d-8f79-71ae0703303e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:06.765445 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:20:06.765355 2567 secret.go:189] Couldn't get secret openshift-monitoring/thanos-querier-grpc-tls-ejih2go9u3e5m: failed to sync secret cache: timed out waiting for the condition Jul 05 13:20:06.765867 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:20:06.765459 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e998c650-79fc-453d-8f79-71ae0703303e-secret-grpc-tls podName:e998c650-79fc-453d-8f79-71ae0703303e nodeName:}" failed. No retries permitted until 2026-07-05 13:20:07.265435551 +0000 UTC m=+93.606453317 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-grpc-tls" (UniqueName: "kubernetes.io/secret/e998c650-79fc-453d-8f79-71ae0703303e-secret-grpc-tls") pod "thanos-querier-78d9c755bc-rnqlg" (UID: "e998c650-79fc-453d-8f79-71ae0703303e") : failed to sync secret cache: timed out waiting for the condition Jul 05 13:20:06.765867 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:20:06.765365 2567 secret.go:189] Couldn't get secret openshift-monitoring/thanos-querier-kube-rbac-proxy-rules: failed to sync secret cache: timed out waiting for the condition Jul 05 13:20:06.765867 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:20:06.765541 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e998c650-79fc-453d-8f79-71ae0703303e-secret-thanos-querier-kube-rbac-proxy-rules podName:e998c650-79fc-453d-8f79-71ae0703303e nodeName:}" failed. No retries permitted until 2026-07-05 13:20:07.265522482 +0000 UTC m=+93.606540234 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-thanos-querier-kube-rbac-proxy-rules" (UniqueName: "kubernetes.io/secret/e998c650-79fc-453d-8f79-71ae0703303e-secret-thanos-querier-kube-rbac-proxy-rules") pod "thanos-querier-78d9c755bc-rnqlg" (UID: "e998c650-79fc-453d-8f79-71ae0703303e") : failed to sync secret cache: timed out waiting for the condition Jul 05 13:20:06.824838 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:06.824801 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 05 13:20:06.902065 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:06.902027 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-t4277\"" Jul 05 13:20:07.137004 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:07.136927 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-ejih2go9u3e5m\"" Jul 05 13:20:07.277143 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:07.277114 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e998c650-79fc-453d-8f79-71ae0703303e-secret-grpc-tls\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:07.277300 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:07.277169 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e998c650-79fc-453d-8f79-71ae0703303e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:07.279634 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:07.279604 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e998c650-79fc-453d-8f79-71ae0703303e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:07.279634 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:07.279615 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e998c650-79fc-453d-8f79-71ae0703303e-secret-grpc-tls\") pod \"thanos-querier-78d9c755bc-rnqlg\" (UID: \"e998c650-79fc-453d-8f79-71ae0703303e\") " pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:07.332144 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:07.332116 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:07.508356 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:07.508330 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-78d9c755bc-rnqlg"] Jul 05 13:20:07.510054 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:20:07.510020 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode998c650_79fc_453d_8f79_71ae0703303e.slice/crio-c93ede9a1a1afc46ff446fdc1152a0df48f864506aa3cda8e060bb490bd4c4b3 WatchSource:0}: Error finding container c93ede9a1a1afc46ff446fdc1152a0df48f864506aa3cda8e060bb490bd4c4b3: Status 404 returned error can't find the container with id c93ede9a1a1afc46ff446fdc1152a0df48f864506aa3cda8e060bb490bd4c4b3 Jul 05 13:20:07.582062 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:07.582035 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:20:07.693677 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:07.693644 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c19f332e-4d92-4078-942b-dd95ef711981","Type":"ContainerStarted","Data":"69ed1c661021a9231e44eadf3a9826d737e6f4c70a543c5bb9c531307738d673"} Jul 05 13:20:07.693874 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:07.693716 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c19f332e-4d92-4078-942b-dd95ef711981","Type":"ContainerStarted","Data":"21ee1b24b8e1f047e9aa5fb47c70922499ceb24848d7be8d381c4d3338a87c13"} Jul 05 13:20:07.693874 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:07.693781 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c19f332e-4d92-4078-942b-dd95ef711981","Type":"ContainerStarted","Data":"a767eb37a1518f46e543f78c7abe556eec7c60031f203c1969bacd9ff5176e50"} Jul 05 13:20:07.693874 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:07.693811 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c19f332e-4d92-4078-942b-dd95ef711981","Type":"ContainerStarted","Data":"43b037bbc469249da713e51eedefebb0d8e713285d533c9f386a105a36c8b564"} Jul 05 13:20:07.693874 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:07.693835 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c19f332e-4d92-4078-942b-dd95ef711981","Type":"ContainerStarted","Data":"7590e883b80e9fffe61340e4aa1eba6cf22c5c5e8cf471db368488c3470a64e4"} Jul 05 13:20:07.694861 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:07.694837 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" event={"ID":"e998c650-79fc-453d-8f79-71ae0703303e","Type":"ContainerStarted","Data":"c93ede9a1a1afc46ff446fdc1152a0df48f864506aa3cda8e060bb490bd4c4b3"} Jul 05 13:20:08.713251 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:08.712799 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c19f332e-4d92-4078-942b-dd95ef711981","Type":"ContainerStarted","Data":"30afa204c15e03d99b0c3269b8251159f5d35d3398c4629850d6207ac6f9967a"} Jul 05 13:20:08.769521 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:08.769459 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.388530827 podStartE2EDuration="5.769435092s" totalImagesPullingTime="3.363913897s" totalInitContainerRuntime="16.990368ms" isStatefulPod=false podCreationTimestamp="2026-07-05 13:20:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:08.764694798 +0000 UTC m=+95.105712569" watchObservedRunningTime="2026-07-05 13:20:08.769435092 +0000 UTC m=+95.110452864" Jul 05 13:20:08.999880 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:08.999805 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 13:20:09.003359 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.003335 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.008766 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.008550 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 05 13:20:09.008766 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.008687 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 05 13:20:09.008958 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.008777 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 05 13:20:09.009092 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.009071 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1rfus47e9tfda\"" Jul 05 13:20:09.009661 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.009506 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 05 13:20:09.009661 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.009514 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 05 13:20:09.009861 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.009671 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 05 13:20:09.009861 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.009755 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 05 13:20:09.009861 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.009780 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 05 13:20:09.010012 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.009885 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 05 13:20:09.010584 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.010564 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 05 13:20:09.011438 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.011420 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 05 13:20:09.012962 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.012944 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zdqqv\"" Jul 05 13:20:09.024767 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.024719 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 05 13:20:09.024955 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.024937 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 05 13:20:09.031083 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.031058 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 05 13:20:09.031261 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.031241 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 05 13:20:09.046969 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.046937 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 13:20:09.093675 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.093637 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.093675 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.093680 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.093932 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.093720 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.093932 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.093802 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.093932 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.093855 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.093932 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.093921 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.094117 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.093964 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-config-out\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.094117 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.093989 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.094117 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.094005 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.094117 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.094056 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.094117 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.094083 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.094117 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.094105 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-web-config\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.094416 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.094128 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-config\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.094416 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.094144 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.094416 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.094160 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.094416 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.094196 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.094416 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.094273 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.094416 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.094311 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.094416 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.094336 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx6t8\" (UniqueName: \"kubernetes.io/projected/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-kube-api-access-rx6t8\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.094416 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.094364 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.194854 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.194814 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.194854 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.194858 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.195092 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.194882 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx6t8\" (UniqueName: \"kubernetes.io/projected/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-kube-api-access-rx6t8\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.195092 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.194913 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.195092 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.194941 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.195092 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.194962 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.195092 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.195002 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.195092 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.195033 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.195092 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.195075 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.195421 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.195102 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.195421 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.195142 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-config-out\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.195421 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.195173 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.195421 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.195194 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.195421 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.195247 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.195421 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.195290 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.195421 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.195313 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-web-config\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.195421 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.195346 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-config\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.195421 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.195370 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.195421 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.195394 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.195421 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.195424 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.196219 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.196192 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.196357 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.196233 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.200474 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.196695 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.200474 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.197339 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.200474 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.198030 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.200474 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.198851 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.200474 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.199915 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.200474 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.200423 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-web-config\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.200863 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.200578 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.200863 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.200702 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.200970 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.200884 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.201508 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.201467 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.201865 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.201833 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-config\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.202108 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.202088 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.202193 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.202144 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-config-out\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.202523 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.202430 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.202610 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.202587 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.203939 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.203913 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.204699 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.204677 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.211208 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.211188 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx6t8\" (UniqueName: \"kubernetes.io/projected/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-kube-api-access-rx6t8\") pod \"prometheus-k8s-0\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.314665 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.313678 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:09.817901 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:09.817869 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 13:20:09.818339 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:20:09.818314 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod491bcc2d_1e26_46e6_93b1_5e50a1ecd963.slice/crio-7d136af6e444961fbb7e849c1d2e1ef688b789d73381d4a59c724c9f9df440f4 WatchSource:0}: Error finding container 7d136af6e444961fbb7e849c1d2e1ef688b789d73381d4a59c724c9f9df440f4: Status 404 returned error can't find the container with id 7d136af6e444961fbb7e849c1d2e1ef688b789d73381d4a59c724c9f9df440f4 Jul 05 13:20:10.725705 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:10.725659 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" event={"ID":"e998c650-79fc-453d-8f79-71ae0703303e","Type":"ContainerStarted","Data":"be987699de716c6e64cee5a94b1171ba9d015b1d73a4bd3d6d1469ac9d02e6fb"} Jul 05 13:20:10.725927 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:10.725795 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:10.725927 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:10.725846 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" event={"ID":"e998c650-79fc-453d-8f79-71ae0703303e","Type":"ContainerStarted","Data":"828f9d023dece71bc13d506491689f0f8754f4f2a825b888b44138c338fdd6ea"} Jul 05 13:20:10.725927 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:10.725886 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" event={"ID":"e998c650-79fc-453d-8f79-71ae0703303e","Type":"ContainerStarted","Data":"c71b788c7fef836964faae84450e1d8494a49421f1af067ee726fdc6f3e2a178"} Jul 05 13:20:10.725927 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:10.725920 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" event={"ID":"e998c650-79fc-453d-8f79-71ae0703303e","Type":"ContainerStarted","Data":"bd3b30be5388db8ea7bd489eb6e3a73faf5d834fe52f4e5b948b0b746d191dd8"} Jul 05 13:20:10.726103 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:10.725958 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" event={"ID":"e998c650-79fc-453d-8f79-71ae0703303e","Type":"ContainerStarted","Data":"cbe8bc59a8bf0e7ae4e765acba7d580f20a09361eb80aa0eb7ef2721039caebb"} Jul 05 13:20:10.726103 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:10.726000 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" event={"ID":"e998c650-79fc-453d-8f79-71ae0703303e","Type":"ContainerStarted","Data":"b8df323ad73a595cb462ae8bccbfa8f2750ee3df180fa84bc947c1e583b8b74d"} Jul 05 13:20:10.727280 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:10.727253 2567 generic.go:356] "Generic (PLEG): container finished" podID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerID="072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e" exitCode=0 Jul 05 13:20:10.727408 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:10.727295 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"491bcc2d-1e26-46e6-93b1-5e50a1ecd963","Type":"ContainerDied","Data":"072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e"} Jul 05 13:20:10.727408 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:10.727353 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"491bcc2d-1e26-46e6-93b1-5e50a1ecd963","Type":"ContainerStarted","Data":"7d136af6e444961fbb7e849c1d2e1ef688b789d73381d4a59c724c9f9df440f4"} Jul 05 13:20:10.807857 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:10.807806 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" podStartSLOduration=3.659542839 podStartE2EDuration="5.807788575s" totalImagesPullingTime="2.148245736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:20:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:10.758618021 +0000 UTC m=+97.099635792" watchObservedRunningTime="2026-07-05 13:20:10.807788575 +0000 UTC m=+97.148806346" Jul 05 13:20:12.542399 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:12.542366 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-j5rvd" Jul 05 13:20:13.300584 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:13.300549 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7986657c49-hk22t"] Jul 05 13:20:14.754870 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:14.754812 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"491bcc2d-1e26-46e6-93b1-5e50a1ecd963","Type":"ContainerStarted","Data":"26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4"} Jul 05 13:20:14.755343 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:14.754883 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"491bcc2d-1e26-46e6-93b1-5e50a1ecd963","Type":"ContainerStarted","Data":"03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589"} Jul 05 13:20:15.763077 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:15.763039 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"491bcc2d-1e26-46e6-93b1-5e50a1ecd963","Type":"ContainerStarted","Data":"ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e"} Jul 05 13:20:15.763548 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:15.763100 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"491bcc2d-1e26-46e6-93b1-5e50a1ecd963","Type":"ContainerStarted","Data":"282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed"} Jul 05 13:20:15.763548 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:15.763125 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"491bcc2d-1e26-46e6-93b1-5e50a1ecd963","Type":"ContainerStarted","Data":"53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b"} Jul 05 13:20:15.763548 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:15.763151 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"491bcc2d-1e26-46e6-93b1-5e50a1ecd963","Type":"ContainerStarted","Data":"5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6"} Jul 05 13:20:16.740037 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:16.740007 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-78d9c755bc-rnqlg" Jul 05 13:20:16.783227 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:16.783173 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.963297076 podStartE2EDuration="8.78315615s" totalImagesPullingTime="3.80455923s" totalInitContainerRuntime="15.299844ms" isStatefulPod=false podCreationTimestamp="2026-07-05 13:20:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:15.807485057 +0000 UTC m=+102.148502829" watchObservedRunningTime="2026-07-05 13:20:16.78315615 +0000 UTC m=+103.124173922" Jul 05 13:20:19.314926 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:19.314891 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:32.836484 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:32.836449 2567 generic.go:356] "Generic (PLEG): container finished" podID="e21b2ebc-6b56-45d0-ab65-e144505dd4b4" containerID="d6f20dbb7568f5e40c83c127a09adff2ba4021e61a55249a8ecfdad6c50d7bc1" exitCode=0 Jul 05 13:20:32.836910 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:32.836528 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rgmdx" event={"ID":"e21b2ebc-6b56-45d0-ab65-e144505dd4b4","Type":"ContainerDied","Data":"d6f20dbb7568f5e40c83c127a09adff2ba4021e61a55249a8ecfdad6c50d7bc1"} Jul 05 13:20:32.836959 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:32.836912 2567 scope.go:117] "RemoveContainer" containerID="d6f20dbb7568f5e40c83c127a09adff2ba4021e61a55249a8ecfdad6c50d7bc1" Jul 05 13:20:33.842482 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:33.842449 2567 generic.go:356] "Generic (PLEG): container finished" podID="532a0da4-c9ae-4e33-96e8-381c6715d7e9" containerID="50b64770e9ee6f268aefdd0991725b391fda1c80d2de1ca153e64839c5f0c742" exitCode=0 Jul 05 13:20:33.842976 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:33.842529 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-7vrt7" event={"ID":"532a0da4-c9ae-4e33-96e8-381c6715d7e9","Type":"ContainerDied","Data":"50b64770e9ee6f268aefdd0991725b391fda1c80d2de1ca153e64839c5f0c742"} Jul 05 13:20:33.842976 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:33.842908 2567 scope.go:117] "RemoveContainer" containerID="50b64770e9ee6f268aefdd0991725b391fda1c80d2de1ca153e64839c5f0c742" Jul 05 13:20:33.844608 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:33.844586 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rgmdx" event={"ID":"e21b2ebc-6b56-45d0-ab65-e144505dd4b4","Type":"ContainerStarted","Data":"922a4bf6a7e67c2a171a20f8fd0bc3093ae23f6e388e4c754bbbddf99e8f7efd"} Jul 05 13:20:34.850387 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:34.850355 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-7vrt7" event={"ID":"532a0da4-c9ae-4e33-96e8-381c6715d7e9","Type":"ContainerStarted","Data":"e6deecf68a2a936d92c8c14bd26137afd6a5cabccd8e36937b158b372b1a8ac2"} Jul 05 13:20:38.325734 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.325694 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7986657c49-hk22t" podUID="b1ce72ed-9388-4861-a28b-4f223346551b" containerName="registry" containerID="cri-o://8cd098222ea5fe4c583b2a9de28fe2764dc2a446eb8822f71ca8d1e59216b091" gracePeriod=30 Jul 05 13:20:38.579051 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.578977 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:20:38.682720 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.682681 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mnbb5\" (UniqueName: \"kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-kube-api-access-mnbb5\") pod \"b1ce72ed-9388-4861-a28b-4f223346551b\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " Jul 05 13:20:38.682932 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.682765 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-bound-sa-token\") pod \"b1ce72ed-9388-4861-a28b-4f223346551b\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " Jul 05 13:20:38.682932 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.682804 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-registry-tls\") pod \"b1ce72ed-9388-4861-a28b-4f223346551b\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " Jul 05 13:20:38.682932 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.682828 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b1ce72ed-9388-4861-a28b-4f223346551b-installation-pull-secrets\") pod \"b1ce72ed-9388-4861-a28b-4f223346551b\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " Jul 05 13:20:38.682932 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.682878 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b1ce72ed-9388-4861-a28b-4f223346551b-image-registry-private-configuration\") pod \"b1ce72ed-9388-4861-a28b-4f223346551b\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " Jul 05 13:20:38.682932 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.682903 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b1ce72ed-9388-4861-a28b-4f223346551b-trusted-ca\") pod \"b1ce72ed-9388-4861-a28b-4f223346551b\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " Jul 05 13:20:38.683177 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.682942 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b1ce72ed-9388-4861-a28b-4f223346551b-registry-certificates\") pod \"b1ce72ed-9388-4861-a28b-4f223346551b\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " Jul 05 13:20:38.683177 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.682991 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b1ce72ed-9388-4861-a28b-4f223346551b-ca-trust-extracted\") pod \"b1ce72ed-9388-4861-a28b-4f223346551b\" (UID: \"b1ce72ed-9388-4861-a28b-4f223346551b\") " Jul 05 13:20:38.683563 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.683503 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1ce72ed-9388-4861-a28b-4f223346551b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "b1ce72ed-9388-4861-a28b-4f223346551b" (UID: "b1ce72ed-9388-4861-a28b-4f223346551b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:38.683665 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.683595 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1ce72ed-9388-4861-a28b-4f223346551b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "b1ce72ed-9388-4861-a28b-4f223346551b" (UID: "b1ce72ed-9388-4861-a28b-4f223346551b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:38.685902 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.685867 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b1ce72ed-9388-4861-a28b-4f223346551b" (UID: "b1ce72ed-9388-4861-a28b-4f223346551b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:20:38.686036 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.685954 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-kube-api-access-mnbb5" (OuterVolumeSpecName: "kube-api-access-mnbb5") pod "b1ce72ed-9388-4861-a28b-4f223346551b" (UID: "b1ce72ed-9388-4861-a28b-4f223346551b"). InnerVolumeSpecName "kube-api-access-mnbb5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:20:38.686036 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.685991 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1ce72ed-9388-4861-a28b-4f223346551b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "b1ce72ed-9388-4861-a28b-4f223346551b" (UID: "b1ce72ed-9388-4861-a28b-4f223346551b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:20:38.686158 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.686032 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "b1ce72ed-9388-4861-a28b-4f223346551b" (UID: "b1ce72ed-9388-4861-a28b-4f223346551b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:20:38.686206 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.686165 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1ce72ed-9388-4861-a28b-4f223346551b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "b1ce72ed-9388-4861-a28b-4f223346551b" (UID: "b1ce72ed-9388-4861-a28b-4f223346551b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:20:38.691464 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.691435 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1ce72ed-9388-4861-a28b-4f223346551b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "b1ce72ed-9388-4861-a28b-4f223346551b" (UID: "b1ce72ed-9388-4861-a28b-4f223346551b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:20:38.784531 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.784485 2567 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-bound-sa-token\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:20:38.784531 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.784521 2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-registry-tls\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:20:38.784531 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.784530 2567 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b1ce72ed-9388-4861-a28b-4f223346551b-installation-pull-secrets\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:20:38.784531 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.784539 2567 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b1ce72ed-9388-4861-a28b-4f223346551b-image-registry-private-configuration\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:20:38.784840 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.784550 2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b1ce72ed-9388-4861-a28b-4f223346551b-trusted-ca\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:20:38.784840 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.784559 2567 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b1ce72ed-9388-4861-a28b-4f223346551b-registry-certificates\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:20:38.784840 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.784567 2567 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b1ce72ed-9388-4861-a28b-4f223346551b-ca-trust-extracted\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:20:38.784840 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.784575 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnbb5\" (UniqueName: \"kubernetes.io/projected/b1ce72ed-9388-4861-a28b-4f223346551b-kube-api-access-mnbb5\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:20:38.867971 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.867882 2567 generic.go:356] "Generic (PLEG): container finished" podID="b1ce72ed-9388-4861-a28b-4f223346551b" containerID="8cd098222ea5fe4c583b2a9de28fe2764dc2a446eb8822f71ca8d1e59216b091" exitCode=0 Jul 05 13:20:38.867971 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.867923 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7986657c49-hk22t" Jul 05 13:20:38.868144 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.867972 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7986657c49-hk22t" event={"ID":"b1ce72ed-9388-4861-a28b-4f223346551b","Type":"ContainerDied","Data":"8cd098222ea5fe4c583b2a9de28fe2764dc2a446eb8822f71ca8d1e59216b091"} Jul 05 13:20:38.868144 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.868022 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7986657c49-hk22t" event={"ID":"b1ce72ed-9388-4861-a28b-4f223346551b","Type":"ContainerDied","Data":"544d003b61e5f3e950ab3bf8a67c5ee2798ae6bafb0476858e95e20cc1dabc09"} Jul 05 13:20:38.868144 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.868044 2567 scope.go:117] "RemoveContainer" containerID="8cd098222ea5fe4c583b2a9de28fe2764dc2a446eb8822f71ca8d1e59216b091" Jul 05 13:20:38.877387 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.877369 2567 scope.go:117] "RemoveContainer" containerID="8cd098222ea5fe4c583b2a9de28fe2764dc2a446eb8822f71ca8d1e59216b091" Jul 05 13:20:38.877725 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:20:38.877694 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cd098222ea5fe4c583b2a9de28fe2764dc2a446eb8822f71ca8d1e59216b091\": container with ID starting with 8cd098222ea5fe4c583b2a9de28fe2764dc2a446eb8822f71ca8d1e59216b091 not found: ID does not exist" containerID="8cd098222ea5fe4c583b2a9de28fe2764dc2a446eb8822f71ca8d1e59216b091" Jul 05 13:20:38.877863 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.877758 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cd098222ea5fe4c583b2a9de28fe2764dc2a446eb8822f71ca8d1e59216b091"} err="failed to get container status \"8cd098222ea5fe4c583b2a9de28fe2764dc2a446eb8822f71ca8d1e59216b091\": rpc error: code = NotFound desc = could not find container \"8cd098222ea5fe4c583b2a9de28fe2764dc2a446eb8822f71ca8d1e59216b091\": container with ID starting with 8cd098222ea5fe4c583b2a9de28fe2764dc2a446eb8822f71ca8d1e59216b091 not found: ID does not exist" Jul 05 13:20:38.890307 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.890267 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7986657c49-hk22t"] Jul 05 13:20:38.892258 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:38.892233 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7986657c49-hk22t"] Jul 05 13:20:40.215035 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:40.215002 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1ce72ed-9388-4861-a28b-4f223346551b" path="/var/lib/kubelet/pods/b1ce72ed-9388-4861-a28b-4f223346551b/volumes" Jul 05 13:20:47.913407 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:47.913358 2567 generic.go:356] "Generic (PLEG): container finished" podID="98016b6f-895f-4fe5-9607-1d2c7ec2f8a3" containerID="af1cf6a07a4e21e535c92ef60378ea08bab36db8aa7483096361ff9ae1e6eba5" exitCode=0 Jul 05 13:20:47.913977 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:47.913428 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7tlcm" event={"ID":"98016b6f-895f-4fe5-9607-1d2c7ec2f8a3","Type":"ContainerDied","Data":"af1cf6a07a4e21e535c92ef60378ea08bab36db8aa7483096361ff9ae1e6eba5"} Jul 05 13:20:47.913977 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:47.913888 2567 scope.go:117] "RemoveContainer" containerID="af1cf6a07a4e21e535c92ef60378ea08bab36db8aa7483096361ff9ae1e6eba5" Jul 05 13:20:48.921673 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:20:48.920781 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7tlcm" event={"ID":"98016b6f-895f-4fe5-9607-1d2c7ec2f8a3","Type":"ContainerStarted","Data":"ab3620e6ee90ef2cde1068f85e17b4635a841e91d7265ead9ceb6bdced1f4cf0"} Jul 05 13:21:09.315042 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:09.314990 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:09.332593 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:09.332564 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:10.026628 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:10.026597 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:22.895802 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:22.895765 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 05 13:21:22.896304 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:22.896049 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="alertmanager" containerID="cri-o://7590e883b80e9fffe61340e4aa1eba6cf22c5c5e8cf471db368488c3470a64e4" gracePeriod=120 Jul 05 13:21:22.896304 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:22.896126 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="kube-rbac-proxy-metric" containerID="cri-o://69ed1c661021a9231e44eadf3a9826d737e6f4c70a543c5bb9c531307738d673" gracePeriod=120 Jul 05 13:21:22.896304 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:22.896126 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="kube-rbac-proxy-web" containerID="cri-o://a767eb37a1518f46e543f78c7abe556eec7c60031f203c1969bacd9ff5176e50" gracePeriod=120 Jul 05 13:21:22.897009 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:22.896546 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="config-reloader" containerID="cri-o://43b037bbc469249da713e51eedefebb0d8e713285d533c9f386a105a36c8b564" gracePeriod=120 Jul 05 13:21:22.897009 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:22.896182 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="prom-label-proxy" containerID="cri-o://30afa204c15e03d99b0c3269b8251159f5d35d3398c4629850d6207ac6f9967a" gracePeriod=120 Jul 05 13:21:22.897009 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:22.896973 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="kube-rbac-proxy" containerID="cri-o://21ee1b24b8e1f047e9aa5fb47c70922499ceb24848d7be8d381c4d3338a87c13" gracePeriod=120 Jul 05 13:21:23.068310 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:23.068275 2567 generic.go:356] "Generic (PLEG): container finished" podID="c19f332e-4d92-4078-942b-dd95ef711981" containerID="30afa204c15e03d99b0c3269b8251159f5d35d3398c4629850d6207ac6f9967a" exitCode=0 Jul 05 13:21:23.068310 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:23.068304 2567 generic.go:356] "Generic (PLEG): container finished" podID="c19f332e-4d92-4078-942b-dd95ef711981" containerID="21ee1b24b8e1f047e9aa5fb47c70922499ceb24848d7be8d381c4d3338a87c13" exitCode=0 Jul 05 13:21:23.068310 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:23.068310 2567 generic.go:356] "Generic (PLEG): container finished" podID="c19f332e-4d92-4078-942b-dd95ef711981" containerID="43b037bbc469249da713e51eedefebb0d8e713285d533c9f386a105a36c8b564" exitCode=0 Jul 05 13:21:23.068310 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:23.068316 2567 generic.go:356] "Generic (PLEG): container finished" podID="c19f332e-4d92-4078-942b-dd95ef711981" containerID="7590e883b80e9fffe61340e4aa1eba6cf22c5c5e8cf471db368488c3470a64e4" exitCode=0 Jul 05 13:21:23.068636 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:23.068352 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c19f332e-4d92-4078-942b-dd95ef711981","Type":"ContainerDied","Data":"30afa204c15e03d99b0c3269b8251159f5d35d3398c4629850d6207ac6f9967a"} Jul 05 13:21:23.068636 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:23.068464 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c19f332e-4d92-4078-942b-dd95ef711981","Type":"ContainerDied","Data":"21ee1b24b8e1f047e9aa5fb47c70922499ceb24848d7be8d381c4d3338a87c13"} Jul 05 13:21:23.068636 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:23.068503 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c19f332e-4d92-4078-942b-dd95ef711981","Type":"ContainerDied","Data":"43b037bbc469249da713e51eedefebb0d8e713285d533c9f386a105a36c8b564"} Jul 05 13:21:23.068636 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:23.068545 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c19f332e-4d92-4078-942b-dd95ef711981","Type":"ContainerDied","Data":"7590e883b80e9fffe61340e4aa1eba6cf22c5c5e8cf471db368488c3470a64e4"} Jul 05 13:21:24.077602 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.077569 2567 generic.go:356] "Generic (PLEG): container finished" podID="c19f332e-4d92-4078-942b-dd95ef711981" containerID="69ed1c661021a9231e44eadf3a9826d737e6f4c70a543c5bb9c531307738d673" exitCode=0 Jul 05 13:21:24.077602 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.077598 2567 generic.go:356] "Generic (PLEG): container finished" podID="c19f332e-4d92-4078-942b-dd95ef711981" containerID="a767eb37a1518f46e543f78c7abe556eec7c60031f203c1969bacd9ff5176e50" exitCode=0 Jul 05 13:21:24.077996 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.077628 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c19f332e-4d92-4078-942b-dd95ef711981","Type":"ContainerDied","Data":"69ed1c661021a9231e44eadf3a9826d737e6f4c70a543c5bb9c531307738d673"} Jul 05 13:21:24.077996 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.077723 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c19f332e-4d92-4078-942b-dd95ef711981","Type":"ContainerDied","Data":"a767eb37a1518f46e543f78c7abe556eec7c60031f203c1969bacd9ff5176e50"} Jul 05 13:21:24.163941 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.163917 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:24.282241 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.282147 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c19f332e-4d92-4078-942b-dd95ef711981-alertmanager-main-db\") pod \"c19f332e-4d92-4078-942b-dd95ef711981\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " Jul 05 13:21:24.282241 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.282197 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-secret-alertmanager-kube-rbac-proxy\") pod \"c19f332e-4d92-4078-942b-dd95ef711981\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " Jul 05 13:21:24.282241 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.282220 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9pr9\" (UniqueName: \"kubernetes.io/projected/c19f332e-4d92-4078-942b-dd95ef711981-kube-api-access-f9pr9\") pod \"c19f332e-4d92-4078-942b-dd95ef711981\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " Jul 05 13:21:24.282241 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.282237 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-cluster-tls-config\") pod \"c19f332e-4d92-4078-942b-dd95ef711981\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " Jul 05 13:21:24.282593 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.282255 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-secret-alertmanager-main-tls\") pod \"c19f332e-4d92-4078-942b-dd95ef711981\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " Jul 05 13:21:24.282593 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.282286 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-config-volume\") pod \"c19f332e-4d92-4078-942b-dd95ef711981\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " Jul 05 13:21:24.282593 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.282335 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c19f332e-4d92-4078-942b-dd95ef711981-tls-assets\") pod \"c19f332e-4d92-4078-942b-dd95ef711981\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " Jul 05 13:21:24.282593 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.282359 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c19f332e-4d92-4078-942b-dd95ef711981-alertmanager-trusted-ca-bundle\") pod \"c19f332e-4d92-4078-942b-dd95ef711981\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " Jul 05 13:21:24.282593 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.282415 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c19f332e-4d92-4078-942b-dd95ef711981-metrics-client-ca\") pod \"c19f332e-4d92-4078-942b-dd95ef711981\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " Jul 05 13:21:24.282593 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.282459 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-web-config\") pod \"c19f332e-4d92-4078-942b-dd95ef711981\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " Jul 05 13:21:24.282593 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.282487 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-secret-alertmanager-kube-rbac-proxy-web\") pod \"c19f332e-4d92-4078-942b-dd95ef711981\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " Jul 05 13:21:24.282593 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.282537 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c19f332e-4d92-4078-942b-dd95ef711981-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "c19f332e-4d92-4078-942b-dd95ef711981" (UID: "c19f332e-4d92-4078-942b-dd95ef711981"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:21:24.282593 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.282562 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-secret-alertmanager-kube-rbac-proxy-metric\") pod \"c19f332e-4d92-4078-942b-dd95ef711981\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " Jul 05 13:21:24.283243 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.282611 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c19f332e-4d92-4078-942b-dd95ef711981-config-out\") pod \"c19f332e-4d92-4078-942b-dd95ef711981\" (UID: \"c19f332e-4d92-4078-942b-dd95ef711981\") " Jul 05 13:21:24.283243 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.282916 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c19f332e-4d92-4078-942b-dd95ef711981-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "c19f332e-4d92-4078-942b-dd95ef711981" (UID: "c19f332e-4d92-4078-942b-dd95ef711981"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:21:24.283243 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.283212 2567 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c19f332e-4d92-4078-942b-dd95ef711981-alertmanager-main-db\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:24.283243 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.283232 2567 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c19f332e-4d92-4078-942b-dd95ef711981-metrics-client-ca\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:24.283445 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.283305 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c19f332e-4d92-4078-942b-dd95ef711981-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "c19f332e-4d92-4078-942b-dd95ef711981" (UID: "c19f332e-4d92-4078-942b-dd95ef711981"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:21:24.285346 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.285315 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c19f332e-4d92-4078-942b-dd95ef711981-kube-api-access-f9pr9" (OuterVolumeSpecName: "kube-api-access-f9pr9") pod "c19f332e-4d92-4078-942b-dd95ef711981" (UID: "c19f332e-4d92-4078-942b-dd95ef711981"). InnerVolumeSpecName "kube-api-access-f9pr9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:21:24.286087 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.286050 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "c19f332e-4d92-4078-942b-dd95ef711981" (UID: "c19f332e-4d92-4078-942b-dd95ef711981"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:24.286199 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.286155 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "c19f332e-4d92-4078-942b-dd95ef711981" (UID: "c19f332e-4d92-4078-942b-dd95ef711981"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:24.286271 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.286224 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c19f332e-4d92-4078-942b-dd95ef711981-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c19f332e-4d92-4078-942b-dd95ef711981" (UID: "c19f332e-4d92-4078-942b-dd95ef711981"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:21:24.286705 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.286671 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-config-volume" (OuterVolumeSpecName: "config-volume") pod "c19f332e-4d92-4078-942b-dd95ef711981" (UID: "c19f332e-4d92-4078-942b-dd95ef711981"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:24.287217 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.287190 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "c19f332e-4d92-4078-942b-dd95ef711981" (UID: "c19f332e-4d92-4078-942b-dd95ef711981"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:24.287355 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.287332 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c19f332e-4d92-4078-942b-dd95ef711981-config-out" (OuterVolumeSpecName: "config-out") pod "c19f332e-4d92-4078-942b-dd95ef711981" (UID: "c19f332e-4d92-4078-942b-dd95ef711981"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:21:24.288425 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.288403 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "c19f332e-4d92-4078-942b-dd95ef711981" (UID: "c19f332e-4d92-4078-942b-dd95ef711981"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:24.290296 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.290266 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "c19f332e-4d92-4078-942b-dd95ef711981" (UID: "c19f332e-4d92-4078-942b-dd95ef711981"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:24.298338 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.298314 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-web-config" (OuterVolumeSpecName: "web-config") pod "c19f332e-4d92-4078-942b-dd95ef711981" (UID: "c19f332e-4d92-4078-942b-dd95ef711981"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:24.384122 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.384083 2567 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c19f332e-4d92-4078-942b-dd95ef711981-tls-assets\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:24.384122 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.384118 2567 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c19f332e-4d92-4078-942b-dd95ef711981-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:24.384122 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.384129 2567 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-web-config\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:24.384371 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.384138 2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:24.384371 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.384149 2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:24.384371 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.384158 2567 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c19f332e-4d92-4078-942b-dd95ef711981-config-out\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:24.384371 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.384167 2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:24.384371 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.384175 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9pr9\" (UniqueName: \"kubernetes.io/projected/c19f332e-4d92-4078-942b-dd95ef711981-kube-api-access-f9pr9\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:24.384371 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.384182 2567 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-cluster-tls-config\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:24.384371 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.384192 2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-secret-alertmanager-main-tls\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:24.384371 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:24.384201 2567 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c19f332e-4d92-4078-942b-dd95ef711981-config-volume\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:25.085177 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.085133 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c19f332e-4d92-4078-942b-dd95ef711981","Type":"ContainerDied","Data":"9bae6fc4a10f4bf237d890d9cb09aaf0f030a5bf9f87d706cc7a4afa50038a65"} Jul 05 13:21:25.085600 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.085194 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.085600 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.085232 2567 scope.go:117] "RemoveContainer" containerID="30afa204c15e03d99b0c3269b8251159f5d35d3398c4629850d6207ac6f9967a" Jul 05 13:21:25.093623 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.093607 2567 scope.go:117] "RemoveContainer" containerID="69ed1c661021a9231e44eadf3a9826d737e6f4c70a543c5bb9c531307738d673" Jul 05 13:21:25.101343 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.101324 2567 scope.go:117] "RemoveContainer" containerID="21ee1b24b8e1f047e9aa5fb47c70922499ceb24848d7be8d381c4d3338a87c13" Jul 05 13:21:25.109267 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.109241 2567 scope.go:117] "RemoveContainer" containerID="a767eb37a1518f46e543f78c7abe556eec7c60031f203c1969bacd9ff5176e50" Jul 05 13:21:25.110959 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.110939 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 05 13:21:25.115610 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.115587 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 05 13:21:25.118589 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.118568 2567 scope.go:117] "RemoveContainer" containerID="43b037bbc469249da713e51eedefebb0d8e713285d533c9f386a105a36c8b564" Jul 05 13:21:25.127438 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.127417 2567 scope.go:117] "RemoveContainer" containerID="7590e883b80e9fffe61340e4aa1eba6cf22c5c5e8cf471db368488c3470a64e4" Jul 05 13:21:25.135448 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.135427 2567 scope.go:117] "RemoveContainer" containerID="f0de24ccaa56ce8d29e867889225d77e4fa9a20704fefd6f87e2bd6dfd659c38" Jul 05 13:21:25.154480 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.154449 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 05 13:21:25.154782 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.154764 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="kube-rbac-proxy-web" Jul 05 13:21:25.154782 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.154780 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="kube-rbac-proxy-web" Jul 05 13:21:25.154955 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.154792 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="alertmanager" Jul 05 13:21:25.154955 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.154799 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="alertmanager" Jul 05 13:21:25.154955 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.154810 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="prom-label-proxy" Jul 05 13:21:25.154955 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.154817 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="prom-label-proxy" Jul 05 13:21:25.154955 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.154828 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="init-config-reloader" Jul 05 13:21:25.154955 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.154836 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="init-config-reloader" Jul 05 13:21:25.154955 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.154845 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="config-reloader" Jul 05 13:21:25.154955 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.154850 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="config-reloader" Jul 05 13:21:25.154955 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.154859 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="kube-rbac-proxy" Jul 05 13:21:25.154955 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.154864 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="kube-rbac-proxy" Jul 05 13:21:25.154955 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.154870 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="kube-rbac-proxy-metric" Jul 05 13:21:25.154955 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.154875 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="kube-rbac-proxy-metric" Jul 05 13:21:25.154955 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.154891 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1ce72ed-9388-4861-a28b-4f223346551b" containerName="registry" Jul 05 13:21:25.154955 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.154899 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1ce72ed-9388-4861-a28b-4f223346551b" containerName="registry" Jul 05 13:21:25.154955 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.154958 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="config-reloader" Jul 05 13:21:25.155662 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.154971 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="kube-rbac-proxy-web" Jul 05 13:21:25.155662 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.154982 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="prom-label-proxy" Jul 05 13:21:25.155662 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.154990 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="kube-rbac-proxy" Jul 05 13:21:25.155662 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.154996 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="alertmanager" Jul 05 13:21:25.155662 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.155003 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1ce72ed-9388-4861-a28b-4f223346551b" containerName="registry" Jul 05 13:21:25.155662 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.155007 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c19f332e-4d92-4078-942b-dd95ef711981" containerName="kube-rbac-proxy-metric" Jul 05 13:21:25.160369 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.160348 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.167647 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.167622 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 05 13:21:25.168155 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.168126 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 05 13:21:25.168155 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.168130 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 05 13:21:25.168863 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.168813 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 05 13:21:25.168994 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.168893 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 05 13:21:25.168994 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.168816 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 05 13:21:25.168994 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.168935 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5ttvt\"" Jul 05 13:21:25.169141 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.168997 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 05 13:21:25.176580 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.176555 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 05 13:21:25.181791 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.181031 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 05 13:21:25.188707 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.188680 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 05 13:21:25.292289 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.292185 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-web-config\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.292289 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.292251 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.292499 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.292314 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.292499 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.292355 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.292499 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.292391 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.292499 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.292411 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-config-out\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.292499 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.292468 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.292499 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.292488 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-config-volume\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.292678 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.292509 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bzfc\" (UniqueName: \"kubernetes.io/projected/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-kube-api-access-6bzfc\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.292678 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.292526 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.292678 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.292585 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.292678 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.292604 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.292678 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.292641 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.393672 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.393638 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bzfc\" (UniqueName: \"kubernetes.io/projected/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-kube-api-access-6bzfc\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.393672 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.393675 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.393924 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.393702 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.393924 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.393721 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.393924 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.393782 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.393924 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.393815 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-web-config\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.393924 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.393857 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.393924 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.393885 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.393924 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.393911 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.394327 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.393946 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.394327 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.393970 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-config-out\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.394327 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.393998 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.394327 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.394018 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-config-volume\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.395304 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.395257 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.397335 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.396981 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-web-config\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.397335 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.397079 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-config-volume\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.397335 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.397122 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.397335 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.397279 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.397587 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.397402 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.397814 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.397791 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.398075 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.398049 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.398371 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.398354 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-config-out\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.398517 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.398500 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.398712 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.398695 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.399731 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.399711 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.403801 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.403781 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bzfc\" (UniqueName: \"kubernetes.io/projected/466e5064-d6a9-4a02-b37f-0d3e47a7fa0e-kube-api-access-6bzfc\") pod \"alertmanager-main-0\" (UID: \"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.469924 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.469884 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:21:25.633244 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:25.633215 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 05 13:21:25.635992 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:21:25.635966 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod466e5064_d6a9_4a02_b37f_0d3e47a7fa0e.slice/crio-270ba46e028664cd906d21922966ac148938b4decb18766120fe7ddd99211bef WatchSource:0}: Error finding container 270ba46e028664cd906d21922966ac148938b4decb18766120fe7ddd99211bef: Status 404 returned error can't find the container with id 270ba46e028664cd906d21922966ac148938b4decb18766120fe7ddd99211bef Jul 05 13:21:26.091535 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:26.091494 2567 generic.go:356] "Generic (PLEG): container finished" podID="466e5064-d6a9-4a02-b37f-0d3e47a7fa0e" containerID="1c8398b7ed3a3cd7af9ac5364fd08097963d413aa394a48e87e67ba8021094ef" exitCode=0 Jul 05 13:21:26.092009 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:26.091590 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e","Type":"ContainerDied","Data":"1c8398b7ed3a3cd7af9ac5364fd08097963d413aa394a48e87e67ba8021094ef"} Jul 05 13:21:26.092009 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:26.091672 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e","Type":"ContainerStarted","Data":"270ba46e028664cd906d21922966ac148938b4decb18766120fe7ddd99211bef"} Jul 05 13:21:26.215815 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:26.215787 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c19f332e-4d92-4078-942b-dd95ef711981" path="/var/lib/kubelet/pods/c19f332e-4d92-4078-942b-dd95ef711981/volumes" Jul 05 13:21:27.102132 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.102095 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e","Type":"ContainerStarted","Data":"3f3d5ff8b8bb2162641055959f18f5f1110eaf2e2bb4ddc8be7c44b3de9448d3"} Jul 05 13:21:27.102630 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.102176 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e","Type":"ContainerStarted","Data":"ccf6c4dafa7e0e446c0eff2b4fbc35e858c1b860c46105406e7996131327be8e"} Jul 05 13:21:27.102630 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.102217 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e","Type":"ContainerStarted","Data":"cdfa918ff33728760a1b5316fd4dae19efe54bba8987b6de8117a3fc90fd35b1"} Jul 05 13:21:27.102630 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.102260 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e","Type":"ContainerStarted","Data":"1e253bbf634a6ce572cbe563b9b869ce7c51770d140eabb7551650f577ada63f"} Jul 05 13:21:27.102630 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.102302 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e","Type":"ContainerStarted","Data":"bb5bbe610a3f9ffb9c88d00e37750751b0cd97f6b116d369f9219cacfadfb301"} Jul 05 13:21:27.102630 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.102347 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"466e5064-d6a9-4a02-b37f-0d3e47a7fa0e","Type":"ContainerStarted","Data":"ac283c0094b7963154d5b97d7db88289dc5928cd1809ef5b5d4209541366384f"} Jul 05 13:21:27.132280 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.132222 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.11668666 podStartE2EDuration="2.13220246s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.5158ms" isStatefulPod=false podCreationTimestamp="2026-07-05 13:21:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:21:27.129684073 +0000 UTC m=+173.470701847" watchObservedRunningTime="2026-07-05 13:21:27.13220246 +0000 UTC m=+173.473220231" Jul 05 13:21:27.235557 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.235522 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 13:21:27.235910 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.235879 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="prometheus" containerID="cri-o://03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589" gracePeriod=600 Jul 05 13:21:27.236136 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.235916 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="thanos-sidecar" containerID="cri-o://ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e" gracePeriod=600 Jul 05 13:21:27.236233 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.235941 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="config-reloader" containerID="cri-o://26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4" gracePeriod=600 Jul 05 13:21:27.236291 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.235944 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="kube-rbac-proxy-web" containerID="cri-o://5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6" gracePeriod=600 Jul 05 13:21:27.236291 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.235901 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="kube-rbac-proxy" containerID="cri-o://53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b" gracePeriod=600 Jul 05 13:21:27.236392 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.235954 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="kube-rbac-proxy-thanos" containerID="cri-o://282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed" gracePeriod=600 Jul 05 13:21:27.504030 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.503240 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:27.617103 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.617008 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-k8s-rulefiles-0\") pod \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " Jul 05 13:21:27.617103 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.617062 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-k8s-rulefiles-1\") pod \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " Jul 05 13:21:27.617103 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.617080 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-metrics-client-certs\") pod \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " Jul 05 13:21:27.617382 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.617106 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-thanos-prometheus-http-client-file\") pod \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " Jul 05 13:21:27.617382 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.617149 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-prometheus-k8s-tls\") pod \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " Jul 05 13:21:27.617382 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.617176 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " Jul 05 13:21:27.617382 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.617208 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-trusted-ca-bundle\") pod \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " Jul 05 13:21:27.617382 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.617251 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-web-config\") pod \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " Jul 05 13:21:27.617382 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.617288 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-k8s-rulefiles-2\") pod \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " Jul 05 13:21:27.617382 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.617314 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-kube-rbac-proxy\") pod \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " Jul 05 13:21:27.617382 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.617343 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-tls-assets\") pod \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " Jul 05 13:21:27.617382 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.617366 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rx6t8\" (UniqueName: \"kubernetes.io/projected/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-kube-api-access-rx6t8\") pod \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " Jul 05 13:21:27.617853 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.617390 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "491bcc2d-1e26-46e6-93b1-5e50a1ecd963" (UID: "491bcc2d-1e26-46e6-93b1-5e50a1ecd963"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:21:27.617853 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.617402 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-configmap-kubelet-serving-ca-bundle\") pod \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " Jul 05 13:21:27.617853 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.617501 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-configmap-serving-certs-ca-bundle\") pod \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " Jul 05 13:21:27.617853 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.617537 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-grpc-tls\") pod \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " Jul 05 13:21:27.617853 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.617588 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-config-out\") pod \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " Jul 05 13:21:27.617853 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.617612 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " Jul 05 13:21:27.617853 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.617639 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-config\") pod \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " Jul 05 13:21:27.617853 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.617671 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-k8s-db\") pod \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " Jul 05 13:21:27.617853 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.617697 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-configmap-metrics-client-ca\") pod \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\" (UID: \"491bcc2d-1e26-46e6-93b1-5e50a1ecd963\") " Jul 05 13:21:27.617853 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.617842 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "491bcc2d-1e26-46e6-93b1-5e50a1ecd963" (UID: "491bcc2d-1e26-46e6-93b1-5e50a1ecd963"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:21:27.618333 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.618199 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "491bcc2d-1e26-46e6-93b1-5e50a1ecd963" (UID: "491bcc2d-1e26-46e6-93b1-5e50a1ecd963"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:21:27.618333 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.618207 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:27.618333 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.618248 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:27.618518 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.618493 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "491bcc2d-1e26-46e6-93b1-5e50a1ecd963" (UID: "491bcc2d-1e26-46e6-93b1-5e50a1ecd963"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:21:27.619346 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.618815 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "491bcc2d-1e26-46e6-93b1-5e50a1ecd963" (UID: "491bcc2d-1e26-46e6-93b1-5e50a1ecd963"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:21:27.619576 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.618850 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "491bcc2d-1e26-46e6-93b1-5e50a1ecd963" (UID: "491bcc2d-1e26-46e6-93b1-5e50a1ecd963"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:21:27.619576 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.619099 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "491bcc2d-1e26-46e6-93b1-5e50a1ecd963" (UID: "491bcc2d-1e26-46e6-93b1-5e50a1ecd963"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:21:27.620513 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.620476 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "491bcc2d-1e26-46e6-93b1-5e50a1ecd963" (UID: "491bcc2d-1e26-46e6-93b1-5e50a1ecd963"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:27.620816 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.620674 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "491bcc2d-1e26-46e6-93b1-5e50a1ecd963" (UID: "491bcc2d-1e26-46e6-93b1-5e50a1ecd963"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:27.620816 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.620782 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "491bcc2d-1e26-46e6-93b1-5e50a1ecd963" (UID: "491bcc2d-1e26-46e6-93b1-5e50a1ecd963"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:27.621581 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.621550 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "491bcc2d-1e26-46e6-93b1-5e50a1ecd963" (UID: "491bcc2d-1e26-46e6-93b1-5e50a1ecd963"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:27.621841 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.621816 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "491bcc2d-1e26-46e6-93b1-5e50a1ecd963" (UID: "491bcc2d-1e26-46e6-93b1-5e50a1ecd963"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:21:27.622155 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.622115 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-config" (OuterVolumeSpecName: "config") pod "491bcc2d-1e26-46e6-93b1-5e50a1ecd963" (UID: "491bcc2d-1e26-46e6-93b1-5e50a1ecd963"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:27.622381 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.622350 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-config-out" (OuterVolumeSpecName: "config-out") pod "491bcc2d-1e26-46e6-93b1-5e50a1ecd963" (UID: "491bcc2d-1e26-46e6-93b1-5e50a1ecd963"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:21:27.622381 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.622366 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "491bcc2d-1e26-46e6-93b1-5e50a1ecd963" (UID: "491bcc2d-1e26-46e6-93b1-5e50a1ecd963"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:27.622522 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.622418 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "491bcc2d-1e26-46e6-93b1-5e50a1ecd963" (UID: "491bcc2d-1e26-46e6-93b1-5e50a1ecd963"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:27.622626 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.622597 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-kube-api-access-rx6t8" (OuterVolumeSpecName: "kube-api-access-rx6t8") pod "491bcc2d-1e26-46e6-93b1-5e50a1ecd963" (UID: "491bcc2d-1e26-46e6-93b1-5e50a1ecd963"). InnerVolumeSpecName "kube-api-access-rx6t8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:21:27.623090 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.623074 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "491bcc2d-1e26-46e6-93b1-5e50a1ecd963" (UID: "491bcc2d-1e26-46e6-93b1-5e50a1ecd963"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:21:27.624079 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.624056 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "491bcc2d-1e26-46e6-93b1-5e50a1ecd963" (UID: "491bcc2d-1e26-46e6-93b1-5e50a1ecd963"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:27.632859 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.632830 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-web-config" (OuterVolumeSpecName: "web-config") pod "491bcc2d-1e26-46e6-93b1-5e50a1ecd963" (UID: "491bcc2d-1e26-46e6-93b1-5e50a1ecd963"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:27.719529 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.719473 2567 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-config\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:27.719529 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.719524 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-k8s-db\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:27.719529 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.719535 2567 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-configmap-metrics-client-ca\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:27.719529 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.719546 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:27.719843 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.719556 2567 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-metrics-client-certs\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:27.719843 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.719565 2567 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:27.719843 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.719573 2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:27.719843 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.719583 2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:27.719843 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.719592 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:27.719843 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.719600 2567 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-web-config\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:27.719843 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.719608 2567 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-kube-rbac-proxy\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:27.719843 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.719619 2567 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-tls-assets\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:27.719843 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.719627 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rx6t8\" (UniqueName: \"kubernetes.io/projected/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-kube-api-access-rx6t8\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:27.719843 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.719635 2567 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:27.719843 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.719643 2567 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:27.719843 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.719652 2567 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-grpc-tls\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:27.719843 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.719660 2567 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-config-out\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:27.719843 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:27.719667 2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/491bcc2d-1e26-46e6-93b1-5e50a1ecd963-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:21:28.109577 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.109538 2567 generic.go:356] "Generic (PLEG): container finished" podID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerID="282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed" exitCode=0 Jul 05 13:21:28.109577 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.109571 2567 generic.go:356] "Generic (PLEG): container finished" podID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerID="53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b" exitCode=0 Jul 05 13:21:28.109577 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.109583 2567 generic.go:356] "Generic (PLEG): container finished" podID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerID="5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6" exitCode=0 Jul 05 13:21:28.110059 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.109594 2567 generic.go:356] "Generic (PLEG): container finished" podID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerID="ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e" exitCode=0 Jul 05 13:21:28.110059 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.109602 2567 generic.go:356] "Generic (PLEG): container finished" podID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerID="26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4" exitCode=0 Jul 05 13:21:28.110059 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.109609 2567 generic.go:356] "Generic (PLEG): container finished" podID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerID="03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589" exitCode=0 Jul 05 13:21:28.110059 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.109632 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.110059 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.109627 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"491bcc2d-1e26-46e6-93b1-5e50a1ecd963","Type":"ContainerDied","Data":"282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed"} Jul 05 13:21:28.110059 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.109729 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"491bcc2d-1e26-46e6-93b1-5e50a1ecd963","Type":"ContainerDied","Data":"53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b"} Jul 05 13:21:28.110059 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.109770 2567 scope.go:117] "RemoveContainer" containerID="282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed" Jul 05 13:21:28.110059 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.109789 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"491bcc2d-1e26-46e6-93b1-5e50a1ecd963","Type":"ContainerDied","Data":"5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6"} Jul 05 13:21:28.110059 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.109828 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"491bcc2d-1e26-46e6-93b1-5e50a1ecd963","Type":"ContainerDied","Data":"ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e"} Jul 05 13:21:28.110059 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.109870 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"491bcc2d-1e26-46e6-93b1-5e50a1ecd963","Type":"ContainerDied","Data":"26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4"} Jul 05 13:21:28.110059 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.109911 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"491bcc2d-1e26-46e6-93b1-5e50a1ecd963","Type":"ContainerDied","Data":"03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589"} Jul 05 13:21:28.110059 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.109947 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"491bcc2d-1e26-46e6-93b1-5e50a1ecd963","Type":"ContainerDied","Data":"7d136af6e444961fbb7e849c1d2e1ef688b789d73381d4a59c724c9f9df440f4"} Jul 05 13:21:28.118536 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.118516 2567 scope.go:117] "RemoveContainer" containerID="53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b" Jul 05 13:21:28.126913 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.126891 2567 scope.go:117] "RemoveContainer" containerID="5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6" Jul 05 13:21:28.135379 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.135352 2567 scope.go:117] "RemoveContainer" containerID="ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e" Jul 05 13:21:28.138044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.138021 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 13:21:28.143815 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.143787 2567 scope.go:117] "RemoveContainer" containerID="26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4" Jul 05 13:21:28.146913 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.146887 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 13:21:28.152534 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.152518 2567 scope.go:117] "RemoveContainer" containerID="03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589" Jul 05 13:21:28.160964 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.160941 2567 scope.go:117] "RemoveContainer" containerID="072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e" Jul 05 13:21:28.169517 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.169497 2567 scope.go:117] "RemoveContainer" containerID="282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed" Jul 05 13:21:28.169845 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:21:28.169823 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed\": container with ID starting with 282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed not found: ID does not exist" containerID="282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed" Jul 05 13:21:28.169924 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.169856 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed"} err="failed to get container status \"282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed\": rpc error: code = NotFound desc = could not find container \"282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed\": container with ID starting with 282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed not found: ID does not exist" Jul 05 13:21:28.169924 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.169875 2567 scope.go:117] "RemoveContainer" containerID="53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b" Jul 05 13:21:28.170153 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:21:28.170135 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b\": container with ID starting with 53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b not found: ID does not exist" containerID="53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b" Jul 05 13:21:28.170194 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.170159 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b"} err="failed to get container status \"53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b\": rpc error: code = NotFound desc = could not find container \"53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b\": container with ID starting with 53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b not found: ID does not exist" Jul 05 13:21:28.170194 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.170176 2567 scope.go:117] "RemoveContainer" containerID="5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6" Jul 05 13:21:28.170457 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:21:28.170440 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6\": container with ID starting with 5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6 not found: ID does not exist" containerID="5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6" Jul 05 13:21:28.170493 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.170463 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6"} err="failed to get container status \"5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6\": rpc error: code = NotFound desc = could not find container \"5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6\": container with ID starting with 5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6 not found: ID does not exist" Jul 05 13:21:28.170493 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.170477 2567 scope.go:117] "RemoveContainer" containerID="ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e" Jul 05 13:21:28.170693 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:21:28.170679 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e\": container with ID starting with ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e not found: ID does not exist" containerID="ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e" Jul 05 13:21:28.170732 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.170696 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e"} err="failed to get container status \"ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e\": rpc error: code = NotFound desc = could not find container \"ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e\": container with ID starting with ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e not found: ID does not exist" Jul 05 13:21:28.170732 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.170711 2567 scope.go:117] "RemoveContainer" containerID="26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4" Jul 05 13:21:28.170961 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:21:28.170946 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4\": container with ID starting with 26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4 not found: ID does not exist" containerID="26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4" Jul 05 13:21:28.171000 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.170967 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4"} err="failed to get container status \"26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4\": rpc error: code = NotFound desc = could not find container \"26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4\": container with ID starting with 26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4 not found: ID does not exist" Jul 05 13:21:28.171000 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.170980 2567 scope.go:117] "RemoveContainer" containerID="03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589" Jul 05 13:21:28.171153 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:21:28.171131 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589\": container with ID starting with 03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589 not found: ID does not exist" containerID="03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589" Jul 05 13:21:28.171215 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.171160 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589"} err="failed to get container status \"03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589\": rpc error: code = NotFound desc = could not find container \"03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589\": container with ID starting with 03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589 not found: ID does not exist" Jul 05 13:21:28.171215 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.171173 2567 scope.go:117] "RemoveContainer" containerID="072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e" Jul 05 13:21:28.171414 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:21:28.171399 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e\": container with ID starting with 072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e not found: ID does not exist" containerID="072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e" Jul 05 13:21:28.171461 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.171416 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e"} err="failed to get container status \"072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e\": rpc error: code = NotFound desc = could not find container \"072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e\": container with ID starting with 072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e not found: ID does not exist" Jul 05 13:21:28.171461 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.171431 2567 scope.go:117] "RemoveContainer" containerID="282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed" Jul 05 13:21:28.171627 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.171614 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed"} err="failed to get container status \"282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed\": rpc error: code = NotFound desc = could not find container \"282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed\": container with ID starting with 282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed not found: ID does not exist" Jul 05 13:21:28.171664 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.171627 2567 scope.go:117] "RemoveContainer" containerID="53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b" Jul 05 13:21:28.171835 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.171818 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b"} err="failed to get container status \"53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b\": rpc error: code = NotFound desc = could not find container \"53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b\": container with ID starting with 53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b not found: ID does not exist" Jul 05 13:21:28.171889 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.171837 2567 scope.go:117] "RemoveContainer" containerID="5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6" Jul 05 13:21:28.172006 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.171992 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6"} err="failed to get container status \"5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6\": rpc error: code = NotFound desc = could not find container \"5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6\": container with ID starting with 5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6 not found: ID does not exist" Jul 05 13:21:28.172006 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.172005 2567 scope.go:117] "RemoveContainer" containerID="ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e" Jul 05 13:21:28.172194 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.172178 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e"} err="failed to get container status \"ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e\": rpc error: code = NotFound desc = could not find container \"ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e\": container with ID starting with ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e not found: ID does not exist" Jul 05 13:21:28.172230 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.172194 2567 scope.go:117] "RemoveContainer" containerID="26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4" Jul 05 13:21:28.172386 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.172370 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4"} err="failed to get container status \"26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4\": rpc error: code = NotFound desc = could not find container \"26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4\": container with ID starting with 26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4 not found: ID does not exist" Jul 05 13:21:28.172427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.172386 2567 scope.go:117] "RemoveContainer" containerID="03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589" Jul 05 13:21:28.172557 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.172544 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589"} err="failed to get container status \"03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589\": rpc error: code = NotFound desc = could not find container \"03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589\": container with ID starting with 03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589 not found: ID does not exist" Jul 05 13:21:28.172557 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.172557 2567 scope.go:117] "RemoveContainer" containerID="072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e" Jul 05 13:21:28.172771 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.172731 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e"} err="failed to get container status \"072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e\": rpc error: code = NotFound desc = could not find container \"072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e\": container with ID starting with 072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e not found: ID does not exist" Jul 05 13:21:28.172771 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.172768 2567 scope.go:117] "RemoveContainer" containerID="282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed" Jul 05 13:21:28.172971 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.172955 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed"} err="failed to get container status \"282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed\": rpc error: code = NotFound desc = could not find container \"282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed\": container with ID starting with 282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed not found: ID does not exist" Jul 05 13:21:28.172971 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.172971 2567 scope.go:117] "RemoveContainer" containerID="53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b" Jul 05 13:21:28.173175 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.173157 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b"} err="failed to get container status \"53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b\": rpc error: code = NotFound desc = could not find container \"53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b\": container with ID starting with 53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b not found: ID does not exist" Jul 05 13:21:28.173245 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.173176 2567 scope.go:117] "RemoveContainer" containerID="5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6" Jul 05 13:21:28.173431 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.173414 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6"} err="failed to get container status \"5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6\": rpc error: code = NotFound desc = could not find container \"5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6\": container with ID starting with 5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6 not found: ID does not exist" Jul 05 13:21:28.173499 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.173433 2567 scope.go:117] "RemoveContainer" containerID="ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e" Jul 05 13:21:28.173660 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.173644 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e"} err="failed to get container status \"ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e\": rpc error: code = NotFound desc = could not find container \"ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e\": container with ID starting with ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e not found: ID does not exist" Jul 05 13:21:28.173718 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.173661 2567 scope.go:117] "RemoveContainer" containerID="26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4" Jul 05 13:21:28.173900 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.173883 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4"} err="failed to get container status \"26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4\": rpc error: code = NotFound desc = could not find container \"26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4\": container with ID starting with 26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4 not found: ID does not exist" Jul 05 13:21:28.173952 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.173902 2567 scope.go:117] "RemoveContainer" containerID="03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589" Jul 05 13:21:28.174123 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.174101 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589"} err="failed to get container status \"03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589\": rpc error: code = NotFound desc = could not find container \"03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589\": container with ID starting with 03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589 not found: ID does not exist" Jul 05 13:21:28.174123 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.174119 2567 scope.go:117] "RemoveContainer" containerID="072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e" Jul 05 13:21:28.174303 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.174287 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e"} err="failed to get container status \"072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e\": rpc error: code = NotFound desc = could not find container \"072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e\": container with ID starting with 072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e not found: ID does not exist" Jul 05 13:21:28.174349 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.174303 2567 scope.go:117] "RemoveContainer" containerID="282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed" Jul 05 13:21:28.174489 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.174474 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed"} err="failed to get container status \"282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed\": rpc error: code = NotFound desc = could not find container \"282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed\": container with ID starting with 282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed not found: ID does not exist" Jul 05 13:21:28.174528 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.174489 2567 scope.go:117] "RemoveContainer" containerID="53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b" Jul 05 13:21:28.174681 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.174662 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b"} err="failed to get container status \"53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b\": rpc error: code = NotFound desc = could not find container \"53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b\": container with ID starting with 53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b not found: ID does not exist" Jul 05 13:21:28.174765 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.174681 2567 scope.go:117] "RemoveContainer" containerID="5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6" Jul 05 13:21:28.174932 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.174906 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6"} err="failed to get container status \"5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6\": rpc error: code = NotFound desc = could not find container \"5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6\": container with ID starting with 5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6 not found: ID does not exist" Jul 05 13:21:28.174932 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.174931 2567 scope.go:117] "RemoveContainer" containerID="ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e" Jul 05 13:21:28.175171 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.175146 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e"} err="failed to get container status \"ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e\": rpc error: code = NotFound desc = could not find container \"ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e\": container with ID starting with ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e not found: ID does not exist" Jul 05 13:21:28.175223 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.175172 2567 scope.go:117] "RemoveContainer" containerID="26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4" Jul 05 13:21:28.175381 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.175363 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4"} err="failed to get container status \"26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4\": rpc error: code = NotFound desc = could not find container \"26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4\": container with ID starting with 26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4 not found: ID does not exist" Jul 05 13:21:28.175420 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.175382 2567 scope.go:117] "RemoveContainer" containerID="03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589" Jul 05 13:21:28.175583 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.175568 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589"} err="failed to get container status \"03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589\": rpc error: code = NotFound desc = could not find container \"03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589\": container with ID starting with 03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589 not found: ID does not exist" Jul 05 13:21:28.175628 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.175583 2567 scope.go:117] "RemoveContainer" containerID="072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e" Jul 05 13:21:28.175838 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.175821 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e"} err="failed to get container status \"072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e\": rpc error: code = NotFound desc = could not find container \"072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e\": container with ID starting with 072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e not found: ID does not exist" Jul 05 13:21:28.175902 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.175839 2567 scope.go:117] "RemoveContainer" containerID="282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed" Jul 05 13:21:28.176078 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.176060 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed"} err="failed to get container status \"282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed\": rpc error: code = NotFound desc = could not find container \"282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed\": container with ID starting with 282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed not found: ID does not exist" Jul 05 13:21:28.176116 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.176078 2567 scope.go:117] "RemoveContainer" containerID="53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b" Jul 05 13:21:28.176279 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.176263 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b"} err="failed to get container status \"53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b\": rpc error: code = NotFound desc = could not find container \"53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b\": container with ID starting with 53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b not found: ID does not exist" Jul 05 13:21:28.176320 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.176279 2567 scope.go:117] "RemoveContainer" containerID="5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6" Jul 05 13:21:28.176490 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.176474 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6"} err="failed to get container status \"5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6\": rpc error: code = NotFound desc = could not find container \"5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6\": container with ID starting with 5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6 not found: ID does not exist" Jul 05 13:21:28.176534 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.176490 2567 scope.go:117] "RemoveContainer" containerID="ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e" Jul 05 13:21:28.176690 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.176676 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e"} err="failed to get container status \"ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e\": rpc error: code = NotFound desc = could not find container \"ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e\": container with ID starting with ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e not found: ID does not exist" Jul 05 13:21:28.176726 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.176690 2567 scope.go:117] "RemoveContainer" containerID="26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4" Jul 05 13:21:28.176917 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.176900 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4"} err="failed to get container status \"26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4\": rpc error: code = NotFound desc = could not find container \"26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4\": container with ID starting with 26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4 not found: ID does not exist" Jul 05 13:21:28.177082 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.176919 2567 scope.go:117] "RemoveContainer" containerID="03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589" Jul 05 13:21:28.177216 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.177186 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589"} err="failed to get container status \"03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589\": rpc error: code = NotFound desc = could not find container \"03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589\": container with ID starting with 03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589 not found: ID does not exist" Jul 05 13:21:28.177313 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.177218 2567 scope.go:117] "RemoveContainer" containerID="072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e" Jul 05 13:21:28.177564 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.177543 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e"} err="failed to get container status \"072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e\": rpc error: code = NotFound desc = could not find container \"072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e\": container with ID starting with 072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e not found: ID does not exist" Jul 05 13:21:28.177659 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.177565 2567 scope.go:117] "RemoveContainer" containerID="282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed" Jul 05 13:21:28.177838 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.177819 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed"} err="failed to get container status \"282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed\": rpc error: code = NotFound desc = could not find container \"282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed\": container with ID starting with 282f3e159b1808308eabd63a82bb4f425f57eea1b298449d850ba93e03f016ed not found: ID does not exist" Jul 05 13:21:28.177926 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.177839 2567 scope.go:117] "RemoveContainer" containerID="53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b" Jul 05 13:21:28.178097 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.178071 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b"} err="failed to get container status \"53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b\": rpc error: code = NotFound desc = could not find container \"53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b\": container with ID starting with 53e4720ae1c2ca1e14d2cca9f484dcf6913d2d03d41d554ab36fa4742154ef4b not found: ID does not exist" Jul 05 13:21:28.178150 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.178105 2567 scope.go:117] "RemoveContainer" containerID="5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6" Jul 05 13:21:28.178330 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.178311 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6"} err="failed to get container status \"5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6\": rpc error: code = NotFound desc = could not find container \"5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6\": container with ID starting with 5afc27d6775e1df5db02857e3b7f97d5d511c8fc766991b4243677e0838438a6 not found: ID does not exist" Jul 05 13:21:28.178410 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.178331 2567 scope.go:117] "RemoveContainer" containerID="ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e" Jul 05 13:21:28.178606 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.178554 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e"} err="failed to get container status \"ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e\": rpc error: code = NotFound desc = could not find container \"ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e\": container with ID starting with ad5a2f8ac088efb4fa1b7c7b216cca821887008e3608a210b8b5b6ec1d70582e not found: ID does not exist" Jul 05 13:21:28.178606 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.178573 2567 scope.go:117] "RemoveContainer" containerID="26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4" Jul 05 13:21:28.178829 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.178810 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4"} err="failed to get container status \"26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4\": rpc error: code = NotFound desc = could not find container \"26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4\": container with ID starting with 26676df9accb6c722fd57c46632a0d1f8695c09e324d0ab944b3164213be16c4 not found: ID does not exist" Jul 05 13:21:28.178829 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.178830 2567 scope.go:117] "RemoveContainer" containerID="03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589" Jul 05 13:21:28.179052 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.179033 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589"} err="failed to get container status \"03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589\": rpc error: code = NotFound desc = could not find container \"03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589\": container with ID starting with 03a2c7e3e1ea01a2bab8f7ccb50d6c987cc7f9e32089c955536d44e18f1a8589 not found: ID does not exist" Jul 05 13:21:28.179129 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.179054 2567 scope.go:117] "RemoveContainer" containerID="072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e" Jul 05 13:21:28.179218 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.179196 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 13:21:28.179282 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.179264 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e"} err="failed to get container status \"072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e\": rpc error: code = NotFound desc = could not find container \"072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e\": container with ID starting with 072f3e37035c04216d1086bc35ec5b63731eee65fb70e314cb58cb483bf02f7e not found: ID does not exist" Jul 05 13:21:28.179553 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.179541 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="kube-rbac-proxy-thanos" Jul 05 13:21:28.179587 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.179557 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="kube-rbac-proxy-thanos" Jul 05 13:21:28.179587 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.179565 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="thanos-sidecar" Jul 05 13:21:28.179587 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.179571 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="thanos-sidecar" Jul 05 13:21:28.179587 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.179586 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="prometheus" Jul 05 13:21:28.179751 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.179592 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="prometheus" Jul 05 13:21:28.179751 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.179598 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="config-reloader" Jul 05 13:21:28.179751 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.179603 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="config-reloader" Jul 05 13:21:28.179751 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.179610 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="init-config-reloader" Jul 05 13:21:28.179751 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.179615 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="init-config-reloader" Jul 05 13:21:28.179751 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.179630 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="kube-rbac-proxy" Jul 05 13:21:28.179751 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.179634 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="kube-rbac-proxy" Jul 05 13:21:28.179751 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.179641 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="kube-rbac-proxy-web" Jul 05 13:21:28.179751 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.179647 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="kube-rbac-proxy-web" Jul 05 13:21:28.179751 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.179703 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="kube-rbac-proxy" Jul 05 13:21:28.179751 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.179713 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="config-reloader" Jul 05 13:21:28.179751 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.179719 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="prometheus" Jul 05 13:21:28.179751 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.179725 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="thanos-sidecar" Jul 05 13:21:28.179751 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.179730 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="kube-rbac-proxy-web" Jul 05 13:21:28.179751 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.179754 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" containerName="kube-rbac-proxy-thanos" Jul 05 13:21:28.185305 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.185287 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.187948 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.187924 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 05 13:21:28.187948 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.187939 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zdqqv\"" Jul 05 13:21:28.188121 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.188104 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 05 13:21:28.188257 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.188231 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 05 13:21:28.188431 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.188418 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 05 13:21:28.188431 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.188428 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 05 13:21:28.188539 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.188473 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 05 13:21:28.188584 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.188548 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 05 13:21:28.189292 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.189276 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 05 13:21:28.189381 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.189358 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 05 13:21:28.189876 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.189859 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 05 13:21:28.190573 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.190323 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 05 13:21:28.190573 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.190389 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 05 13:21:28.190573 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.190426 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1rfus47e9tfda\"" Jul 05 13:21:28.190573 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.190480 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 05 13:21:28.191180 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.191161 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 05 13:21:28.194708 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.194679 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 05 13:21:28.201455 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.201430 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 13:21:28.216174 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.216139 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="491bcc2d-1e26-46e6-93b1-5e50a1ecd963" path="/var/lib/kubelet/pods/491bcc2d-1e26-46e6-93b1-5e50a1ecd963/volumes" Jul 05 13:21:28.324407 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.324310 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.324407 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.324353 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd65a853-f61d-481b-ba5f-7ac4d667beb4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.324407 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.324387 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd65a853-f61d-481b-ba5f-7ac4d667beb4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.324618 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.324423 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cd65a853-f61d-481b-ba5f-7ac4d667beb4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.324618 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.324451 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cd65a853-f61d-481b-ba5f-7ac4d667beb4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.324618 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.324492 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-web-config\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.324618 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.324514 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.324618 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.324529 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.324618 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.324543 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.324874 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.324642 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-config\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.324874 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.324675 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd65a853-f61d-481b-ba5f-7ac4d667beb4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.324874 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.324700 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd65a853-f61d-481b-ba5f-7ac4d667beb4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.324874 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.324725 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cd65a853-f61d-481b-ba5f-7ac4d667beb4-config-out\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.324874 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.324773 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cd65a853-f61d-481b-ba5f-7ac4d667beb4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.324874 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.324824 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cd65a853-f61d-481b-ba5f-7ac4d667beb4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.324874 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.324853 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.325064 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.324879 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgll5\" (UniqueName: \"kubernetes.io/projected/cd65a853-f61d-481b-ba5f-7ac4d667beb4-kube-api-access-mgll5\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.325064 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.324898 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.325064 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.324919 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.325064 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.324965 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cd65a853-f61d-481b-ba5f-7ac4d667beb4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.425877 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.425832 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-web-config\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.425877 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.425881 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.426114 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.425901 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.426114 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.425925 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.426114 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.425956 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-config\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.426114 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.425979 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd65a853-f61d-481b-ba5f-7ac4d667beb4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.426114 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.426006 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd65a853-f61d-481b-ba5f-7ac4d667beb4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.426114 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.426038 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cd65a853-f61d-481b-ba5f-7ac4d667beb4-config-out\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.426510 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.426170 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cd65a853-f61d-481b-ba5f-7ac4d667beb4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.426510 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.426455 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cd65a853-f61d-481b-ba5f-7ac4d667beb4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.426603 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.426509 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.426603 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.426538 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgll5\" (UniqueName: \"kubernetes.io/projected/cd65a853-f61d-481b-ba5f-7ac4d667beb4-kube-api-access-mgll5\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.426603 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.426571 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.426781 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.426607 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.426781 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.426643 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cd65a853-f61d-481b-ba5f-7ac4d667beb4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.426781 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.426696 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.426781 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.426725 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd65a853-f61d-481b-ba5f-7ac4d667beb4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.427004 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.426783 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd65a853-f61d-481b-ba5f-7ac4d667beb4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.427004 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.426812 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cd65a853-f61d-481b-ba5f-7ac4d667beb4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.427004 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.426844 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cd65a853-f61d-481b-ba5f-7ac4d667beb4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.427004 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.426949 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd65a853-f61d-481b-ba5f-7ac4d667beb4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.427004 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.426951 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd65a853-f61d-481b-ba5f-7ac4d667beb4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.427255 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.427184 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cd65a853-f61d-481b-ba5f-7ac4d667beb4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.429277 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.429110 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.429393 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.429356 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.429393 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.429358 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cd65a853-f61d-481b-ba5f-7ac4d667beb4-config-out\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.429496 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.429356 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-config\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.429615 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.429538 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-web-config\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.430424 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.430030 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd65a853-f61d-481b-ba5f-7ac4d667beb4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.430424 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.430152 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cd65a853-f61d-481b-ba5f-7ac4d667beb4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.430424 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.430223 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.430424 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.430316 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd65a853-f61d-481b-ba5f-7ac4d667beb4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.430690 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.430526 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.430934 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.430841 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cd65a853-f61d-481b-ba5f-7ac4d667beb4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.431378 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.431350 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cd65a853-f61d-481b-ba5f-7ac4d667beb4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.431993 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.431966 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.432208 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.432189 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.432579 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.432560 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cd65a853-f61d-481b-ba5f-7ac4d667beb4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.433208 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.433190 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cd65a853-f61d-481b-ba5f-7ac4d667beb4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.439497 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.439475 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgll5\" (UniqueName: \"kubernetes.io/projected/cd65a853-f61d-481b-ba5f-7ac4d667beb4-kube-api-access-mgll5\") pod \"prometheus-k8s-0\" (UID: \"cd65a853-f61d-481b-ba5f-7ac4d667beb4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.496337 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.496292 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:28.650557 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:28.650523 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 13:21:28.653680 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:21:28.653656 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd65a853_f61d_481b_ba5f_7ac4d667beb4.slice/crio-196231e100057e13ca319e3f592a590e648e8b59668c1cdfe8eed8c7e5f57d79 WatchSource:0}: Error finding container 196231e100057e13ca319e3f592a590e648e8b59668c1cdfe8eed8c7e5f57d79: Status 404 returned error can't find the container with id 196231e100057e13ca319e3f592a590e648e8b59668c1cdfe8eed8c7e5f57d79 Jul 05 13:21:29.116353 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:29.116312 2567 generic.go:356] "Generic (PLEG): container finished" podID="cd65a853-f61d-481b-ba5f-7ac4d667beb4" containerID="f14ff649c6de30c6787d1aa503c1bc4d074d785b894832496eaea34872a4396b" exitCode=0 Jul 05 13:21:29.116790 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:29.116395 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cd65a853-f61d-481b-ba5f-7ac4d667beb4","Type":"ContainerDied","Data":"f14ff649c6de30c6787d1aa503c1bc4d074d785b894832496eaea34872a4396b"} Jul 05 13:21:29.116790 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:29.116462 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cd65a853-f61d-481b-ba5f-7ac4d667beb4","Type":"ContainerStarted","Data":"196231e100057e13ca319e3f592a590e648e8b59668c1cdfe8eed8c7e5f57d79"} Jul 05 13:21:30.125404 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:30.125362 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cd65a853-f61d-481b-ba5f-7ac4d667beb4","Type":"ContainerStarted","Data":"baf42b9684da6693935398bd75d128bab3074cf6fdd46b0be50a1a3c4aea44a2"} Jul 05 13:21:30.125785 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:30.125426 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cd65a853-f61d-481b-ba5f-7ac4d667beb4","Type":"ContainerStarted","Data":"73d88b6c3637cd5130adf03eb2dfb7ca94951525a48617be49d121575fb3db5b"} Jul 05 13:21:30.125785 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:30.125453 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cd65a853-f61d-481b-ba5f-7ac4d667beb4","Type":"ContainerStarted","Data":"e0aba3782f20c4212a7dc1a7a92f037da8d8143bc0ebfcab21a46c12f4c74059"} Jul 05 13:21:30.125785 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:30.125483 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cd65a853-f61d-481b-ba5f-7ac4d667beb4","Type":"ContainerStarted","Data":"21d9e8f757b51d89468b603456f56b50ebb6d44dc3b8c5d2501b1ebf1f815e0e"} Jul 05 13:21:30.125785 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:30.125511 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cd65a853-f61d-481b-ba5f-7ac4d667beb4","Type":"ContainerStarted","Data":"6b09d5bd2ebd93a184962f8ababc311cd7b752fbe1d6ab7652625c7b14de9a97"} Jul 05 13:21:30.125785 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:30.125534 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cd65a853-f61d-481b-ba5f-7ac4d667beb4","Type":"ContainerStarted","Data":"c19efd972f70aac8025665dd68b989c9bfd5b5ae34b9edc92d322900b827a5b6"} Jul 05 13:21:30.161916 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:30.161858 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.139446154 podStartE2EDuration="2.161839587s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.393433ms" isStatefulPod=false podCreationTimestamp="2026-07-05 13:21:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:21:30.15909071 +0000 UTC m=+176.500108482" watchObservedRunningTime="2026-07-05 13:21:30.161839587 +0000 UTC m=+176.502857359" Jul 05 13:21:33.496860 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:21:33.496822 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:22:28.496656 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:22:28.496605 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:22:28.513404 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:22:28.513375 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:22:29.396934 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:22:29.396906 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:23:34.081231 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:23:34.081203 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-26jcf_bccc5e19-26b8-4f6b-96f2-92684dc3d25a/console-operator/1.log" Jul 05 13:23:34.094458 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:23:34.094419 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swckn_a447ce81-b3cb-4269-8d46-e6e612b89e0b/ovn-acl-logging/0.log" Jul 05 13:23:34.095630 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:23:34.095605 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-26jcf_bccc5e19-26b8-4f6b-96f2-92684dc3d25a/console-operator/1.log" Jul 05 13:23:34.099903 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:23:34.099881 2567 kubelet.go:1648] "Image garbage collection succeeded" Jul 05 13:23:34.102710 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:23:34.102691 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swckn_a447ce81-b3cb-4269-8d46-e6e612b89e0b/ovn-acl-logging/0.log" Jul 05 13:24:39.163848 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:24:39.163811 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2wbsv"] Jul 05 13:24:39.167258 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:24:39.167238 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2wbsv" Jul 05 13:24:39.170136 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:24:39.170116 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 05 13:24:39.171237 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:24:39.171220 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-j49f4\"" Jul 05 13:24:39.171237 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:24:39.171227 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:24:39.179953 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:24:39.179917 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2wbsv"] Jul 05 13:24:39.251086 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:24:39.251042 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfhtr\" (UniqueName: \"kubernetes.io/projected/6852ebdd-87cb-48bc-8b94-da3c61fb1b02-kube-api-access-gfhtr\") pod \"openshift-lws-operator-fd8ccff4c-2wbsv\" (UID: \"6852ebdd-87cb-48bc-8b94-da3c61fb1b02\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2wbsv" Jul 05 13:24:39.251270 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:24:39.251173 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6852ebdd-87cb-48bc-8b94-da3c61fb1b02-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2wbsv\" (UID: \"6852ebdd-87cb-48bc-8b94-da3c61fb1b02\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2wbsv" Jul 05 13:24:39.352007 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:24:39.351962 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfhtr\" (UniqueName: \"kubernetes.io/projected/6852ebdd-87cb-48bc-8b94-da3c61fb1b02-kube-api-access-gfhtr\") pod \"openshift-lws-operator-fd8ccff4c-2wbsv\" (UID: \"6852ebdd-87cb-48bc-8b94-da3c61fb1b02\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2wbsv" Jul 05 13:24:39.352007 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:24:39.352008 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6852ebdd-87cb-48bc-8b94-da3c61fb1b02-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2wbsv\" (UID: \"6852ebdd-87cb-48bc-8b94-da3c61fb1b02\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2wbsv" Jul 05 13:24:39.352432 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:24:39.352414 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6852ebdd-87cb-48bc-8b94-da3c61fb1b02-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2wbsv\" (UID: \"6852ebdd-87cb-48bc-8b94-da3c61fb1b02\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2wbsv" Jul 05 13:24:39.363877 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:24:39.363842 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfhtr\" (UniqueName: \"kubernetes.io/projected/6852ebdd-87cb-48bc-8b94-da3c61fb1b02-kube-api-access-gfhtr\") pod \"openshift-lws-operator-fd8ccff4c-2wbsv\" (UID: \"6852ebdd-87cb-48bc-8b94-da3c61fb1b02\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2wbsv" Jul 05 13:24:39.485950 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:24:39.485853 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2wbsv" Jul 05 13:24:39.643566 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:24:39.643541 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2wbsv"] Jul 05 13:24:39.645468 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:24:39.645438 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6852ebdd_87cb_48bc_8b94_da3c61fb1b02.slice/crio-76ec1c2c2d01ed7d0945c0ce39c1aa509a92b96662a0ab5f7afd8152f611e5ad WatchSource:0}: Error finding container 76ec1c2c2d01ed7d0945c0ce39c1aa509a92b96662a0ab5f7afd8152f611e5ad: Status 404 returned error can't find the container with id 76ec1c2c2d01ed7d0945c0ce39c1aa509a92b96662a0ab5f7afd8152f611e5ad Jul 05 13:24:39.646926 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:24:39.646908 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 13:24:39.925207 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:24:39.925172 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2wbsv" event={"ID":"6852ebdd-87cb-48bc-8b94-da3c61fb1b02","Type":"ContainerStarted","Data":"76ec1c2c2d01ed7d0945c0ce39c1aa509a92b96662a0ab5f7afd8152f611e5ad"} Jul 05 13:24:42.941259 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:24:42.941224 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2wbsv" event={"ID":"6852ebdd-87cb-48bc-8b94-da3c61fb1b02","Type":"ContainerStarted","Data":"492845453c67e6e3a6525bcbe10006fbbf132920ef803feaa168e919e4589495"} Jul 05 13:24:42.960965 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:24:42.960915 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2wbsv" podStartSLOduration=1.453366841 podStartE2EDuration="3.960897758s" totalImagesPullingTime="2.507530917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:24:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:24:42.960240011 +0000 UTC m=+369.301257785" watchObservedRunningTime="2026-07-05 13:24:42.960897758 +0000 UTC m=+369.301915530" Jul 05 13:25:03.747969 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:03.747931 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j"] Jul 05 13:25:03.751355 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:03.751337 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" Jul 05 13:25:03.758380 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:25:03.758343 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-manager-service-cert\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" type="*v1.Secret" Jul 05 13:25:03.758525 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:25:03.758384 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"opendatahub-operator-controller-manager-b4f468d94-hm24j\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="cc491ec3-6f95-4406-8abd-2c7a3d665eeb" pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" Jul 05 13:25:03.758525 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:25:03.758407 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 05 13:25:03.758525 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:25:03.758416 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-manager-dockercfg-bmwcw\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-bmwcw\"" type="*v1.Secret" Jul 05 13:25:03.758525 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:25:03.758409 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-webhook-cert\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" type="*v1.Secret" Jul 05 13:25:03.758525 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:25:03.758430 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 05 13:25:03.774321 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:03.774284 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j"] Jul 05 13:25:03.871866 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:03.871829 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cc491ec3-6f95-4406-8abd-2c7a3d665eeb-manifests\") pod \"opendatahub-operator-controller-manager-b4f468d94-hm24j\" (UID: \"cc491ec3-6f95-4406-8abd-2c7a3d665eeb\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" Jul 05 13:25:03.872065 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:03.871877 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc491ec3-6f95-4406-8abd-2c7a3d665eeb-webhook-cert\") pod \"opendatahub-operator-controller-manager-b4f468d94-hm24j\" (UID: \"cc491ec3-6f95-4406-8abd-2c7a3d665eeb\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" Jul 05 13:25:03.872065 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:03.871912 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr9mq\" (UniqueName: \"kubernetes.io/projected/cc491ec3-6f95-4406-8abd-2c7a3d665eeb-kube-api-access-gr9mq\") pod \"opendatahub-operator-controller-manager-b4f468d94-hm24j\" (UID: \"cc491ec3-6f95-4406-8abd-2c7a3d665eeb\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" Jul 05 13:25:03.872065 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:03.871967 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cc491ec3-6f95-4406-8abd-2c7a3d665eeb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b4f468d94-hm24j\" (UID: \"cc491ec3-6f95-4406-8abd-2c7a3d665eeb\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" Jul 05 13:25:03.872065 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:03.872025 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc491ec3-6f95-4406-8abd-2c7a3d665eeb-tmp\") pod \"opendatahub-operator-controller-manager-b4f468d94-hm24j\" (UID: \"cc491ec3-6f95-4406-8abd-2c7a3d665eeb\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" Jul 05 13:25:03.973396 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:03.973357 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cc491ec3-6f95-4406-8abd-2c7a3d665eeb-manifests\") pod \"opendatahub-operator-controller-manager-b4f468d94-hm24j\" (UID: \"cc491ec3-6f95-4406-8abd-2c7a3d665eeb\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" Jul 05 13:25:03.973396 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:03.973403 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc491ec3-6f95-4406-8abd-2c7a3d665eeb-webhook-cert\") pod \"opendatahub-operator-controller-manager-b4f468d94-hm24j\" (UID: \"cc491ec3-6f95-4406-8abd-2c7a3d665eeb\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" Jul 05 13:25:03.973659 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:03.973421 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr9mq\" (UniqueName: \"kubernetes.io/projected/cc491ec3-6f95-4406-8abd-2c7a3d665eeb-kube-api-access-gr9mq\") pod \"opendatahub-operator-controller-manager-b4f468d94-hm24j\" (UID: \"cc491ec3-6f95-4406-8abd-2c7a3d665eeb\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" Jul 05 13:25:03.973659 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:03.973563 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cc491ec3-6f95-4406-8abd-2c7a3d665eeb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b4f468d94-hm24j\" (UID: \"cc491ec3-6f95-4406-8abd-2c7a3d665eeb\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" Jul 05 13:25:03.973659 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:03.973635 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc491ec3-6f95-4406-8abd-2c7a3d665eeb-tmp\") pod \"opendatahub-operator-controller-manager-b4f468d94-hm24j\" (UID: \"cc491ec3-6f95-4406-8abd-2c7a3d665eeb\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" Jul 05 13:25:03.973900 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:03.973878 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cc491ec3-6f95-4406-8abd-2c7a3d665eeb-manifests\") pod \"opendatahub-operator-controller-manager-b4f468d94-hm24j\" (UID: \"cc491ec3-6f95-4406-8abd-2c7a3d665eeb\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" Jul 05 13:25:03.976043 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:03.976022 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc491ec3-6f95-4406-8abd-2c7a3d665eeb-tmp\") pod \"opendatahub-operator-controller-manager-b4f468d94-hm24j\" (UID: \"cc491ec3-6f95-4406-8abd-2c7a3d665eeb\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" Jul 05 13:25:04.810626 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:04.810586 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 05 13:25:04.813874 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:04.813852 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 05 13:25:04.838024 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:04.837994 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 05 13:25:04.841558 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:04.841533 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr9mq\" (UniqueName: \"kubernetes.io/projected/cc491ec3-6f95-4406-8abd-2c7a3d665eeb-kube-api-access-gr9mq\") pod \"opendatahub-operator-controller-manager-b4f468d94-hm24j\" (UID: \"cc491ec3-6f95-4406-8abd-2c7a3d665eeb\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" Jul 05 13:25:04.962981 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:04.962949 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-bmwcw\"" Jul 05 13:25:04.973635 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:25:04.973612 2567 secret.go:189] Couldn't get secret opendatahub/opendatahub-operator-controller-manager-service-cert: failed to sync secret cache: timed out waiting for the condition Jul 05 13:25:04.973761 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:25:04.973696 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc491ec3-6f95-4406-8abd-2c7a3d665eeb-webhook-cert podName:cc491ec3-6f95-4406-8abd-2c7a3d665eeb nodeName:}" failed. No retries permitted until 2026-07-05 13:25:05.4736769 +0000 UTC m=+391.814694649 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/cc491ec3-6f95-4406-8abd-2c7a3d665eeb-webhook-cert") pod "opendatahub-operator-controller-manager-b4f468d94-hm24j" (UID: "cc491ec3-6f95-4406-8abd-2c7a3d665eeb") : failed to sync secret cache: timed out waiting for the condition Jul 05 13:25:04.973816 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:25:04.973784 2567 secret.go:189] Couldn't get secret opendatahub/opendatahub-operator-controller-manager-service-cert: failed to sync secret cache: timed out waiting for the condition Jul 05 13:25:04.973874 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:25:04.973861 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc491ec3-6f95-4406-8abd-2c7a3d665eeb-apiservice-cert podName:cc491ec3-6f95-4406-8abd-2c7a3d665eeb nodeName:}" failed. No retries permitted until 2026-07-05 13:25:05.473841754 +0000 UTC m=+391.814859508 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "apiservice-cert" (UniqueName: "kubernetes.io/secret/cc491ec3-6f95-4406-8abd-2c7a3d665eeb-apiservice-cert") pod "opendatahub-operator-controller-manager-b4f468d94-hm24j" (UID: "cc491ec3-6f95-4406-8abd-2c7a3d665eeb") : failed to sync secret cache: timed out waiting for the condition Jul 05 13:25:05.275729 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:05.275695 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 05 13:25:05.486500 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:05.486458 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc491ec3-6f95-4406-8abd-2c7a3d665eeb-webhook-cert\") pod \"opendatahub-operator-controller-manager-b4f468d94-hm24j\" (UID: \"cc491ec3-6f95-4406-8abd-2c7a3d665eeb\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" Jul 05 13:25:05.486701 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:05.486528 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cc491ec3-6f95-4406-8abd-2c7a3d665eeb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b4f468d94-hm24j\" (UID: \"cc491ec3-6f95-4406-8abd-2c7a3d665eeb\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" Jul 05 13:25:05.489220 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:05.489186 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc491ec3-6f95-4406-8abd-2c7a3d665eeb-webhook-cert\") pod \"opendatahub-operator-controller-manager-b4f468d94-hm24j\" (UID: \"cc491ec3-6f95-4406-8abd-2c7a3d665eeb\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" Jul 05 13:25:05.489341 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:05.489186 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cc491ec3-6f95-4406-8abd-2c7a3d665eeb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b4f468d94-hm24j\" (UID: \"cc491ec3-6f95-4406-8abd-2c7a3d665eeb\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" Jul 05 13:25:05.561689 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:05.561587 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" Jul 05 13:25:05.721991 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:05.721949 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j"] Jul 05 13:25:05.724445 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:25:05.724411 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc491ec3_6f95_4406_8abd_2c7a3d665eeb.slice/crio-53de43e2d929d3b0cf3fa92eff6fc667854bb3b67b750e5c1d721000011849f6 WatchSource:0}: Error finding container 53de43e2d929d3b0cf3fa92eff6fc667854bb3b67b750e5c1d721000011849f6: Status 404 returned error can't find the container with id 53de43e2d929d3b0cf3fa92eff6fc667854bb3b67b750e5c1d721000011849f6 Jul 05 13:25:06.040886 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:06.040848 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" event={"ID":"cc491ec3-6f95-4406-8abd-2c7a3d665eeb","Type":"ContainerStarted","Data":"53de43e2d929d3b0cf3fa92eff6fc667854bb3b67b750e5c1d721000011849f6"} Jul 05 13:25:09.058862 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:09.058820 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" event={"ID":"cc491ec3-6f95-4406-8abd-2c7a3d665eeb","Type":"ContainerStarted","Data":"59f1e8d14205b224a8205548a58fd4cd2e0b1b10c6c17173dd1bcfb00fb7b6ee"} Jul 05 13:25:10.064440 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:10.064331 2567 generic.go:356] "Generic (PLEG): container finished" podID="cc491ec3-6f95-4406-8abd-2c7a3d665eeb" containerID="59f1e8d14205b224a8205548a58fd4cd2e0b1b10c6c17173dd1bcfb00fb7b6ee" exitCode=0 Jul 05 13:25:10.064440 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:10.064376 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" event={"ID":"cc491ec3-6f95-4406-8abd-2c7a3d665eeb","Type":"ContainerDied","Data":"59f1e8d14205b224a8205548a58fd4cd2e0b1b10c6c17173dd1bcfb00fb7b6ee"} Jul 05 13:25:11.071519 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:11.071487 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" event={"ID":"cc491ec3-6f95-4406-8abd-2c7a3d665eeb","Type":"ContainerStarted","Data":"cbe902e888882a7d9a7b5d910c23987f4a17ecd67fcc09c2fa0954d4f6e2c456"} Jul 05 13:25:11.071940 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:11.071607 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" Jul 05 13:25:11.126649 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:11.126594 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" podStartSLOduration=3.9163458 podStartE2EDuration="8.126578678s" totalImagesPullingTime="3.172856331s" totalInitContainerRuntime="1.037376547s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:25:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:25:11.126054727 +0000 UTC m=+397.467072499" watchObservedRunningTime="2026-07-05 13:25:11.126578678 +0000 UTC m=+397.467596451" Jul 05 13:25:22.078475 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:22.078444 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-hm24j" Jul 05 13:25:25.219706 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.219664 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-67cd4f9bbf-9q6vw"] Jul 05 13:25:25.223200 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.223177 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-9q6vw" Jul 05 13:25:25.226929 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.226907 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 05 13:25:25.227058 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.226913 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 05 13:25:25.227058 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.226913 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-rpcgq\"" Jul 05 13:25:25.240326 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.240281 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-67cd4f9bbf-9q6vw"] Jul 05 13:25:25.369366 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.369309 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4226538c-272c-4a18-a7aa-6da1545c1907-tls-certs\") pod \"kube-auth-proxy-67cd4f9bbf-9q6vw\" (UID: \"4226538c-272c-4a18-a7aa-6da1545c1907\") " pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-9q6vw" Jul 05 13:25:25.369366 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.369368 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4226538c-272c-4a18-a7aa-6da1545c1907-tmp\") pod \"kube-auth-proxy-67cd4f9bbf-9q6vw\" (UID: \"4226538c-272c-4a18-a7aa-6da1545c1907\") " pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-9q6vw" Jul 05 13:25:25.369689 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.369423 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9dhf\" (UniqueName: \"kubernetes.io/projected/4226538c-272c-4a18-a7aa-6da1545c1907-kube-api-access-w9dhf\") pod \"kube-auth-proxy-67cd4f9bbf-9q6vw\" (UID: \"4226538c-272c-4a18-a7aa-6da1545c1907\") " pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-9q6vw" Jul 05 13:25:25.470283 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.470181 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4226538c-272c-4a18-a7aa-6da1545c1907-tls-certs\") pod \"kube-auth-proxy-67cd4f9bbf-9q6vw\" (UID: \"4226538c-272c-4a18-a7aa-6da1545c1907\") " pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-9q6vw" Jul 05 13:25:25.470283 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.470221 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4226538c-272c-4a18-a7aa-6da1545c1907-tmp\") pod \"kube-auth-proxy-67cd4f9bbf-9q6vw\" (UID: \"4226538c-272c-4a18-a7aa-6da1545c1907\") " pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-9q6vw" Jul 05 13:25:25.470283 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.470254 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9dhf\" (UniqueName: \"kubernetes.io/projected/4226538c-272c-4a18-a7aa-6da1545c1907-kube-api-access-w9dhf\") pod \"kube-auth-proxy-67cd4f9bbf-9q6vw\" (UID: \"4226538c-272c-4a18-a7aa-6da1545c1907\") " pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-9q6vw" Jul 05 13:25:25.472689 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.472655 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4226538c-272c-4a18-a7aa-6da1545c1907-tmp\") pod \"kube-auth-proxy-67cd4f9bbf-9q6vw\" (UID: \"4226538c-272c-4a18-a7aa-6da1545c1907\") " pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-9q6vw" Jul 05 13:25:25.472896 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.472878 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4226538c-272c-4a18-a7aa-6da1545c1907-tls-certs\") pod \"kube-auth-proxy-67cd4f9bbf-9q6vw\" (UID: \"4226538c-272c-4a18-a7aa-6da1545c1907\") " pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-9q6vw" Jul 05 13:25:25.497998 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.497966 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9dhf\" (UniqueName: \"kubernetes.io/projected/4226538c-272c-4a18-a7aa-6da1545c1907-kube-api-access-w9dhf\") pod \"kube-auth-proxy-67cd4f9bbf-9q6vw\" (UID: \"4226538c-272c-4a18-a7aa-6da1545c1907\") " pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-9q6vw" Jul 05 13:25:25.533259 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.533228 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-9q6vw" Jul 05 13:25:25.564083 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.564053 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-86bkx"] Jul 05 13:25:25.567679 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.567662 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-86bkx" Jul 05 13:25:25.574920 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.574896 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-mz5np\"" Jul 05 13:25:25.596657 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.596617 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 05 13:25:25.624487 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.624417 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-86bkx"] Jul 05 13:25:25.671761 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.671709 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/21bf44b7-9381-4cbd-a733-b5d85c4f6bdf-cert\") pod \"odh-model-controller-7998c67dc4-86bkx\" (UID: \"21bf44b7-9381-4cbd-a733-b5d85c4f6bdf\") " pod="opendatahub/odh-model-controller-7998c67dc4-86bkx" Jul 05 13:25:25.671928 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.671796 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gxcc\" (UniqueName: \"kubernetes.io/projected/21bf44b7-9381-4cbd-a733-b5d85c4f6bdf-kube-api-access-5gxcc\") pod \"odh-model-controller-7998c67dc4-86bkx\" (UID: \"21bf44b7-9381-4cbd-a733-b5d85c4f6bdf\") " pod="opendatahub/odh-model-controller-7998c67dc4-86bkx" Jul 05 13:25:25.687614 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.687570 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-67cd4f9bbf-9q6vw"] Jul 05 13:25:25.689163 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:25:25.689138 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4226538c_272c_4a18_a7aa_6da1545c1907.slice/crio-ef1065252f2cba72a86a7e53bc5415d6ea7a80b03765d8db7746e60352b06cd7 WatchSource:0}: Error finding container ef1065252f2cba72a86a7e53bc5415d6ea7a80b03765d8db7746e60352b06cd7: Status 404 returned error can't find the container with id ef1065252f2cba72a86a7e53bc5415d6ea7a80b03765d8db7746e60352b06cd7 Jul 05 13:25:25.772673 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.772576 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/21bf44b7-9381-4cbd-a733-b5d85c4f6bdf-cert\") pod \"odh-model-controller-7998c67dc4-86bkx\" (UID: \"21bf44b7-9381-4cbd-a733-b5d85c4f6bdf\") " pod="opendatahub/odh-model-controller-7998c67dc4-86bkx" Jul 05 13:25:25.772673 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.772634 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gxcc\" (UniqueName: \"kubernetes.io/projected/21bf44b7-9381-4cbd-a733-b5d85c4f6bdf-kube-api-access-5gxcc\") pod \"odh-model-controller-7998c67dc4-86bkx\" (UID: \"21bf44b7-9381-4cbd-a733-b5d85c4f6bdf\") " pod="opendatahub/odh-model-controller-7998c67dc4-86bkx" Jul 05 13:25:25.772886 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:25:25.772727 2567 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 05 13:25:25.772886 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:25:25.772805 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21bf44b7-9381-4cbd-a733-b5d85c4f6bdf-cert podName:21bf44b7-9381-4cbd-a733-b5d85c4f6bdf nodeName:}" failed. No retries permitted until 2026-07-05 13:25:26.272789777 +0000 UTC m=+412.613807527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/21bf44b7-9381-4cbd-a733-b5d85c4f6bdf-cert") pod "odh-model-controller-7998c67dc4-86bkx" (UID: "21bf44b7-9381-4cbd-a733-b5d85c4f6bdf") : secret "odh-model-controller-webhook-cert" not found Jul 05 13:25:25.787218 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:25.787190 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gxcc\" (UniqueName: \"kubernetes.io/projected/21bf44b7-9381-4cbd-a733-b5d85c4f6bdf-kube-api-access-5gxcc\") pod \"odh-model-controller-7998c67dc4-86bkx\" (UID: \"21bf44b7-9381-4cbd-a733-b5d85c4f6bdf\") " pod="opendatahub/odh-model-controller-7998c67dc4-86bkx" Jul 05 13:25:26.144225 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:26.144107 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-9q6vw" event={"ID":"4226538c-272c-4a18-a7aa-6da1545c1907","Type":"ContainerStarted","Data":"ef1065252f2cba72a86a7e53bc5415d6ea7a80b03765d8db7746e60352b06cd7"} Jul 05 13:25:26.278166 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:26.278123 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/21bf44b7-9381-4cbd-a733-b5d85c4f6bdf-cert\") pod \"odh-model-controller-7998c67dc4-86bkx\" (UID: \"21bf44b7-9381-4cbd-a733-b5d85c4f6bdf\") " pod="opendatahub/odh-model-controller-7998c67dc4-86bkx" Jul 05 13:25:26.280942 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:26.280913 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/21bf44b7-9381-4cbd-a733-b5d85c4f6bdf-cert\") pod \"odh-model-controller-7998c67dc4-86bkx\" (UID: \"21bf44b7-9381-4cbd-a733-b5d85c4f6bdf\") " pod="opendatahub/odh-model-controller-7998c67dc4-86bkx" Jul 05 13:25:26.479078 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:26.479035 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-86bkx" Jul 05 13:25:26.739621 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:26.739540 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-86bkx"] Jul 05 13:25:26.746801 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:25:26.746735 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21bf44b7_9381_4cbd_a733_b5d85c4f6bdf.slice/crio-9218773d1d24a40c52382131889bb0da0100edb84560fb49839ca76185df88a3 WatchSource:0}: Error finding container 9218773d1d24a40c52382131889bb0da0100edb84560fb49839ca76185df88a3: Status 404 returned error can't find the container with id 9218773d1d24a40c52382131889bb0da0100edb84560fb49839ca76185df88a3 Jul 05 13:25:27.152811 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:27.152708 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-86bkx" event={"ID":"21bf44b7-9381-4cbd-a733-b5d85c4f6bdf","Type":"ContainerStarted","Data":"9218773d1d24a40c52382131889bb0da0100edb84560fb49839ca76185df88a3"} Jul 05 13:25:27.226684 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:27.226653 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-c4mkr"] Jul 05 13:25:27.230507 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:27.230480 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-c4mkr" Jul 05 13:25:27.234783 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:27.234725 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 05 13:25:27.252538 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:27.252435 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-c4mkr"] Jul 05 13:25:27.389359 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:27.389318 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/912df14c-0f0d-4069-b323-ada776d5fd6f-tls-certs\") pod \"model-serving-api-54d89bd5fc-c4mkr\" (UID: \"912df14c-0f0d-4069-b323-ada776d5fd6f\") " pod="opendatahub/model-serving-api-54d89bd5fc-c4mkr" Jul 05 13:25:27.389852 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:27.389473 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjgd8\" (UniqueName: \"kubernetes.io/projected/912df14c-0f0d-4069-b323-ada776d5fd6f-kube-api-access-sjgd8\") pod \"model-serving-api-54d89bd5fc-c4mkr\" (UID: \"912df14c-0f0d-4069-b323-ada776d5fd6f\") " pod="opendatahub/model-serving-api-54d89bd5fc-c4mkr" Jul 05 13:25:27.490835 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:27.490797 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjgd8\" (UniqueName: \"kubernetes.io/projected/912df14c-0f0d-4069-b323-ada776d5fd6f-kube-api-access-sjgd8\") pod \"model-serving-api-54d89bd5fc-c4mkr\" (UID: \"912df14c-0f0d-4069-b323-ada776d5fd6f\") " pod="opendatahub/model-serving-api-54d89bd5fc-c4mkr" Jul 05 13:25:27.491026 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:27.490876 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/912df14c-0f0d-4069-b323-ada776d5fd6f-tls-certs\") pod \"model-serving-api-54d89bd5fc-c4mkr\" (UID: \"912df14c-0f0d-4069-b323-ada776d5fd6f\") " pod="opendatahub/model-serving-api-54d89bd5fc-c4mkr" Jul 05 13:25:27.491079 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:25:27.491037 2567 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 05 13:25:27.491204 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:25:27.491111 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/912df14c-0f0d-4069-b323-ada776d5fd6f-tls-certs podName:912df14c-0f0d-4069-b323-ada776d5fd6f nodeName:}" failed. No retries permitted until 2026-07-05 13:25:27.991094298 +0000 UTC m=+414.332112048 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/912df14c-0f0d-4069-b323-ada776d5fd6f-tls-certs") pod "model-serving-api-54d89bd5fc-c4mkr" (UID: "912df14c-0f0d-4069-b323-ada776d5fd6f") : secret "model-serving-api-tls" not found Jul 05 13:25:27.510382 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:27.510354 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjgd8\" (UniqueName: \"kubernetes.io/projected/912df14c-0f0d-4069-b323-ada776d5fd6f-kube-api-access-sjgd8\") pod \"model-serving-api-54d89bd5fc-c4mkr\" (UID: \"912df14c-0f0d-4069-b323-ada776d5fd6f\") " pod="opendatahub/model-serving-api-54d89bd5fc-c4mkr" Jul 05 13:25:27.944637 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:27.944589 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn"] Jul 05 13:25:27.948393 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:27.948363 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:27.952062 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:27.952030 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jk5b2\"" Jul 05 13:25:27.952873 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:27.952852 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 05 13:25:27.953007 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:27.952854 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 05 13:25:27.953007 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:27.952904 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 05 13:25:27.962869 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:27.962847 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 05 13:25:27.969499 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:27.969423 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn"] Jul 05 13:25:27.994757 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:27.994708 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/912df14c-0f0d-4069-b323-ada776d5fd6f-tls-certs\") pod \"model-serving-api-54d89bd5fc-c4mkr\" (UID: \"912df14c-0f0d-4069-b323-ada776d5fd6f\") " pod="opendatahub/model-serving-api-54d89bd5fc-c4mkr" Jul 05 13:25:27.998032 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:27.998004 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/912df14c-0f0d-4069-b323-ada776d5fd6f-tls-certs\") pod \"model-serving-api-54d89bd5fc-c4mkr\" (UID: \"912df14c-0f0d-4069-b323-ada776d5fd6f\") " pod="opendatahub/model-serving-api-54d89bd5fc-c4mkr" Jul 05 13:25:28.096286 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.096204 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-c9vhn\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:28.096286 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.096250 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-istio-token\") pod \"istiod-openshift-gateway-7574459656-c9vhn\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:28.096286 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.096268 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-cacerts\") pod \"istiod-openshift-gateway-7574459656-c9vhn\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:28.096581 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.096351 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-c9vhn\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:28.096581 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.096372 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-c9vhn\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:28.096581 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.096506 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-local-certs\") pod \"istiod-openshift-gateway-7574459656-c9vhn\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:28.096581 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.096547 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz297\" (UniqueName: \"kubernetes.io/projected/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-kube-api-access-pz297\") pod \"istiod-openshift-gateway-7574459656-c9vhn\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:28.143285 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.143238 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-c4mkr" Jul 05 13:25:28.163419 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.163290 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-9q6vw" event={"ID":"4226538c-272c-4a18-a7aa-6da1545c1907","Type":"ContainerStarted","Data":"e6e7723dd128a44c02acd7e5deb292cb210b8dc550a6207341c2f48f0b488b2b"} Jul 05 13:25:28.187762 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.187676 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-9q6vw" podStartSLOduration=1.106459532 podStartE2EDuration="3.187652871s" totalImagesPullingTime="2.081193339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:25:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:25:28.184957372 +0000 UTC m=+414.525975146" watchObservedRunningTime="2026-07-05 13:25:28.187652871 +0000 UTC m=+414.528670638" Jul 05 13:25:28.199121 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.198455 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-cacerts\") pod \"istiod-openshift-gateway-7574459656-c9vhn\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:28.199121 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.198513 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-c9vhn\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:28.199121 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.198546 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-c9vhn\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:28.199121 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.198637 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-local-certs\") pod \"istiod-openshift-gateway-7574459656-c9vhn\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:28.199121 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.198665 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz297\" (UniqueName: \"kubernetes.io/projected/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-kube-api-access-pz297\") pod \"istiod-openshift-gateway-7574459656-c9vhn\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:28.199121 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.198699 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-c9vhn\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:28.199121 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.198756 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-istio-token\") pod \"istiod-openshift-gateway-7574459656-c9vhn\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:28.200048 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.199694 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-c9vhn\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:28.213782 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.204473 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-c9vhn\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:28.213782 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.204791 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-local-certs\") pod \"istiod-openshift-gateway-7574459656-c9vhn\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:28.215212 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.215110 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-cacerts\") pod \"istiod-openshift-gateway-7574459656-c9vhn\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:28.220686 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.220150 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-c9vhn\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:28.221170 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.221103 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-istio-token\") pod \"istiod-openshift-gateway-7574459656-c9vhn\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:28.222846 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.222817 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz297\" (UniqueName: \"kubernetes.io/projected/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-kube-api-access-pz297\") pod \"istiod-openshift-gateway-7574459656-c9vhn\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:28.262773 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.262545 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:28.330127 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.330092 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-c4mkr"] Jul 05 13:25:28.456145 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:28.456079 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn"] Jul 05 13:25:28.458383 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:25:28.458355 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode933d34a_8ea4_4483_ac1f_0d0cf1b89e12.slice/crio-e1f3a37298fead0a514f90602bc4d9f2a67cec2163d8c8db240137f3062dc9ce WatchSource:0}: Error finding container e1f3a37298fead0a514f90602bc4d9f2a67cec2163d8c8db240137f3062dc9ce: Status 404 returned error can't find the container with id e1f3a37298fead0a514f90602bc4d9f2a67cec2163d8c8db240137f3062dc9ce Jul 05 13:25:29.171230 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:29.171133 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" event={"ID":"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12","Type":"ContainerStarted","Data":"e1f3a37298fead0a514f90602bc4d9f2a67cec2163d8c8db240137f3062dc9ce"} Jul 05 13:25:29.173223 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:29.173163 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-c4mkr" event={"ID":"912df14c-0f0d-4069-b323-ada776d5fd6f","Type":"ContainerStarted","Data":"b318ca09231c5605b143dd78fb8c93dd3bc34aa2699fd8d1e96c91e8f6d0b0a7"} Jul 05 13:25:31.187267 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:31.187144 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-86bkx" event={"ID":"21bf44b7-9381-4cbd-a733-b5d85c4f6bdf","Type":"ContainerStarted","Data":"f6d5dc16e6b619d23e40dbcaafcbbc411c5ceeb887dd3ee5bf81dee6e9d5d960"} Jul 05 13:25:31.187267 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:31.187222 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-86bkx" Jul 05 13:25:31.189012 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:31.188970 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-c4mkr" event={"ID":"912df14c-0f0d-4069-b323-ada776d5fd6f","Type":"ContainerStarted","Data":"91191212a849442649ca61307e0f4c7d79e04834305c31a71ec2867ed7900137"} Jul 05 13:25:31.189556 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:31.189530 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-c4mkr" Jul 05 13:25:31.225754 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:31.225679 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-86bkx" podStartSLOduration=2.299950961 podStartE2EDuration="6.225646828s" totalImagesPullingTime="3.925695867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:25:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:25:31.224309387 +0000 UTC m=+417.565327142" watchObservedRunningTime="2026-07-05 13:25:31.225646828 +0000 UTC m=+417.566664601" Jul 05 13:25:31.256929 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:31.256875 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-c4mkr" podStartSLOduration=1.9224409599999999 podStartE2EDuration="4.256852092s" totalImagesPullingTime="2.334411132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:25:31.254343789 +0000 UTC m=+417.595361563" watchObservedRunningTime="2026-07-05 13:25:31.256852092 +0000 UTC m=+417.597869867" Jul 05 13:25:31.994870 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:31.994833 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:25:31.994968 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:31.994901 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:25:32.196031 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:32.195991 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" event={"ID":"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12","Type":"ContainerStarted","Data":"17857b09b2458dae792d4ebaeac2f2c188a14de3270ced10a62642d1daa3895c"} Jul 05 13:25:32.196466 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:32.196150 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:32.238709 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:32.238645 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" podStartSLOduration=1.704487126 podStartE2EDuration="5.238620932s" totalImagesPullingTime="3.534133806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:25:32.236663728 +0000 UTC m=+418.577681499" watchObservedRunningTime="2026-07-05 13:25:32.238620932 +0000 UTC m=+418.579638706" Jul 05 13:25:33.204589 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:33.204552 2567 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-c9vhn container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 05 13:25:33.205077 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:33.204610 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" podUID="e933d34a-8ea4-4483-ac1f-0d0cf1b89e12" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 05 13:25:35.494986 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.494947 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2"] Jul 05 13:25:35.500007 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.499984 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.506637 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.506612 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-46bzg\"" Jul 05 13:25:35.529648 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.529612 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2"] Jul 05 13:25:35.573238 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.573200 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrr8l\" (UniqueName: \"kubernetes.io/projected/8ae1e965-9222-4287-8542-1ab0d3b923c5-kube-api-access-hrr8l\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.573443 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.573245 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ae1e965-9222-4287-8542-1ab0d3b923c5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.573443 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.573328 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ae1e965-9222-4287-8542-1ab0d3b923c5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.573443 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.573365 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ae1e965-9222-4287-8542-1ab0d3b923c5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.573443 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.573400 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ae1e965-9222-4287-8542-1ab0d3b923c5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.573443 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.573434 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ae1e965-9222-4287-8542-1ab0d3b923c5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.573638 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.573451 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ae1e965-9222-4287-8542-1ab0d3b923c5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.573638 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.573466 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ae1e965-9222-4287-8542-1ab0d3b923c5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.573638 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.573491 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ae1e965-9222-4287-8542-1ab0d3b923c5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.673941 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.673908 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ae1e965-9222-4287-8542-1ab0d3b923c5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.674113 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.673975 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrr8l\" (UniqueName: \"kubernetes.io/projected/8ae1e965-9222-4287-8542-1ab0d3b923c5-kube-api-access-hrr8l\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.674113 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.674005 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ae1e965-9222-4287-8542-1ab0d3b923c5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.674113 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.674044 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ae1e965-9222-4287-8542-1ab0d3b923c5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.674113 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.674065 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ae1e965-9222-4287-8542-1ab0d3b923c5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.674321 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.674196 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ae1e965-9222-4287-8542-1ab0d3b923c5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.674321 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.674265 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ae1e965-9222-4287-8542-1ab0d3b923c5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.674321 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.674289 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ae1e965-9222-4287-8542-1ab0d3b923c5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.674321 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.674310 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ae1e965-9222-4287-8542-1ab0d3b923c5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.674542 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.674514 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ae1e965-9222-4287-8542-1ab0d3b923c5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.674627 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.674602 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ae1e965-9222-4287-8542-1ab0d3b923c5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.674765 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.674664 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ae1e965-9222-4287-8542-1ab0d3b923c5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.674894 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.674817 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ae1e965-9222-4287-8542-1ab0d3b923c5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.674894 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.674869 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ae1e965-9222-4287-8542-1ab0d3b923c5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.676608 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.676588 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ae1e965-9222-4287-8542-1ab0d3b923c5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.676689 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.676635 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ae1e965-9222-4287-8542-1ab0d3b923c5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.754320 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.754238 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ae1e965-9222-4287-8542-1ab0d3b923c5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.769335 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.769297 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrr8l\" (UniqueName: \"kubernetes.io/projected/8ae1e965-9222-4287-8542-1ab0d3b923c5-kube-api-access-hrr8l\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4pg8k2\" (UID: \"8ae1e965-9222-4287-8542-1ab0d3b923c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.810491 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.810446 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:35.985153 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:25:35.985105 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ae1e965_9222_4287_8542_1ab0d3b923c5.slice/crio-140badc273541f94cfe1d7e98d0197d5fd3531fa15ef5c0baefe6fec56818723 WatchSource:0}: Error finding container 140badc273541f94cfe1d7e98d0197d5fd3531fa15ef5c0baefe6fec56818723: Status 404 returned error can't find the container with id 140badc273541f94cfe1d7e98d0197d5fd3531fa15ef5c0baefe6fec56818723 Jul 05 13:25:35.985153 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:35.985112 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2"] Jul 05 13:25:36.204404 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:36.204371 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:36.219922 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:36.219883 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" event={"ID":"8ae1e965-9222-4287-8542-1ab0d3b923c5","Type":"ContainerStarted","Data":"140badc273541f94cfe1d7e98d0197d5fd3531fa15ef5c0baefe6fec56818723"} Jul 05 13:25:38.930010 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:38.929977 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:25:38.930273 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:38.930050 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:25:38.930273 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:38.930080 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:25:39.237953 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:39.237859 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" event={"ID":"8ae1e965-9222-4287-8542-1ab0d3b923c5","Type":"ContainerStarted","Data":"668c35cd4b2dcbd150a816f9ed9109f7ef062db0087802a020278a9f9ee962eb"} Jul 05 13:25:39.312822 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:39.312765 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" podStartSLOduration=1.370295236 podStartE2EDuration="4.312734022s" totalImagesPullingTime="2.942438786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:25:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:25:39.31123579 +0000 UTC m=+425.652253563" watchObservedRunningTime="2026-07-05 13:25:39.312734022 +0000 UTC m=+425.653751793" Jul 05 13:25:39.810959 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:39.810920 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:39.812410 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:39.812383 2567 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 05 13:25:39.812624 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:39.812455 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" podUID="8ae1e965-9222-4287-8542-1ab0d3b923c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 05 13:25:40.815255 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:40.815218 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:41.200130 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.200090 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-slvsr"] Jul 05 13:25:41.203974 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.203951 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7nx7z"] Jul 05 13:25:41.204103 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.204069 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-slvsr" Jul 05 13:25:41.206914 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.206890 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 05 13:25:41.207337 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.207318 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nx7z" Jul 05 13:25:41.208435 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.208417 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-svbsf\"" Jul 05 13:25:41.214609 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.214587 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-fwvpz\"" Jul 05 13:25:41.216511 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.216493 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 05 13:25:41.225137 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.225115 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jcbx\" (UniqueName: \"kubernetes.io/projected/626bf575-c542-415d-9d9a-8480539083c5-kube-api-access-7jcbx\") pod \"kserve-controller-manager-d6f4b65b8-slvsr\" (UID: \"626bf575-c542-415d-9d9a-8480539083c5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-slvsr" Jul 05 13:25:41.225276 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.225234 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rmvb\" (UniqueName: \"kubernetes.io/projected/83909d91-90dd-4e68-9104-a8d2138b50ec-kube-api-access-5rmvb\") pod \"llmisvc-controller-manager-777c6bd7d5-7nx7z\" (UID: \"83909d91-90dd-4e68-9104-a8d2138b50ec\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nx7z" Jul 05 13:25:41.225343 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.225278 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/83909d91-90dd-4e68-9104-a8d2138b50ec-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7nx7z\" (UID: \"83909d91-90dd-4e68-9104-a8d2138b50ec\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nx7z" Jul 05 13:25:41.225449 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.225428 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/626bf575-c542-415d-9d9a-8480539083c5-cert\") pod \"kserve-controller-manager-d6f4b65b8-slvsr\" (UID: \"626bf575-c542-415d-9d9a-8480539083c5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-slvsr" Jul 05 13:25:41.239056 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.239004 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-slvsr"] Jul 05 13:25:41.241663 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.241631 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7nx7z"] Jul 05 13:25:41.248433 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.248397 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:41.249525 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.249499 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pg8k2" Jul 05 13:25:41.326852 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.326820 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jcbx\" (UniqueName: \"kubernetes.io/projected/626bf575-c542-415d-9d9a-8480539083c5-kube-api-access-7jcbx\") pod \"kserve-controller-manager-d6f4b65b8-slvsr\" (UID: \"626bf575-c542-415d-9d9a-8480539083c5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-slvsr" Jul 05 13:25:41.327056 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.326907 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rmvb\" (UniqueName: \"kubernetes.io/projected/83909d91-90dd-4e68-9104-a8d2138b50ec-kube-api-access-5rmvb\") pod \"llmisvc-controller-manager-777c6bd7d5-7nx7z\" (UID: \"83909d91-90dd-4e68-9104-a8d2138b50ec\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nx7z" Jul 05 13:25:41.327056 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.326941 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/83909d91-90dd-4e68-9104-a8d2138b50ec-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7nx7z\" (UID: \"83909d91-90dd-4e68-9104-a8d2138b50ec\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nx7z" Jul 05 13:25:41.327177 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.327054 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/626bf575-c542-415d-9d9a-8480539083c5-cert\") pod \"kserve-controller-manager-d6f4b65b8-slvsr\" (UID: \"626bf575-c542-415d-9d9a-8480539083c5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-slvsr" Jul 05 13:25:41.327177 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:25:41.327161 2567 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 05 13:25:41.327298 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:25:41.327206 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/626bf575-c542-415d-9d9a-8480539083c5-cert podName:626bf575-c542-415d-9d9a-8480539083c5 nodeName:}" failed. No retries permitted until 2026-07-05 13:25:41.827193363 +0000 UTC m=+428.168211113 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/626bf575-c542-415d-9d9a-8480539083c5-cert") pod "kserve-controller-manager-d6f4b65b8-slvsr" (UID: "626bf575-c542-415d-9d9a-8480539083c5") : secret "kserve-webhook-server-cert" not found Jul 05 13:25:41.329512 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.329492 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/83909d91-90dd-4e68-9104-a8d2138b50ec-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7nx7z\" (UID: \"83909d91-90dd-4e68-9104-a8d2138b50ec\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nx7z" Jul 05 13:25:41.365645 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.365607 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rmvb\" (UniqueName: \"kubernetes.io/projected/83909d91-90dd-4e68-9104-a8d2138b50ec-kube-api-access-5rmvb\") pod \"llmisvc-controller-manager-777c6bd7d5-7nx7z\" (UID: \"83909d91-90dd-4e68-9104-a8d2138b50ec\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nx7z" Jul 05 13:25:41.365830 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.365688 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jcbx\" (UniqueName: \"kubernetes.io/projected/626bf575-c542-415d-9d9a-8480539083c5-kube-api-access-7jcbx\") pod \"kserve-controller-manager-d6f4b65b8-slvsr\" (UID: \"626bf575-c542-415d-9d9a-8480539083c5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-slvsr" Jul 05 13:25:41.522535 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.522448 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nx7z" Jul 05 13:25:41.736373 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.736331 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7nx7z"] Jul 05 13:25:41.772119 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:25:41.772089 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod83909d91_90dd_4e68_9104_a8d2138b50ec.slice/crio-0ef46f322cf8658117ff24c8f660f264463a3006c6fa740249d9871eded710ca WatchSource:0}: Error finding container 0ef46f322cf8658117ff24c8f660f264463a3006c6fa740249d9871eded710ca: Status 404 returned error can't find the container with id 0ef46f322cf8658117ff24c8f660f264463a3006c6fa740249d9871eded710ca Jul 05 13:25:41.832129 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.832096 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/626bf575-c542-415d-9d9a-8480539083c5-cert\") pod \"kserve-controller-manager-d6f4b65b8-slvsr\" (UID: \"626bf575-c542-415d-9d9a-8480539083c5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-slvsr" Jul 05 13:25:41.834606 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:41.834582 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/626bf575-c542-415d-9d9a-8480539083c5-cert\") pod \"kserve-controller-manager-d6f4b65b8-slvsr\" (UID: \"626bf575-c542-415d-9d9a-8480539083c5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-slvsr" Jul 05 13:25:42.093677 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.093590 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn"] Jul 05 13:25:42.093852 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.093791 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" podUID="e933d34a-8ea4-4483-ac1f-0d0cf1b89e12" containerName="discovery" containerID="cri-o://17857b09b2458dae792d4ebaeac2f2c188a14de3270ced10a62642d1daa3895c" gracePeriod=30 Jul 05 13:25:42.116416 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.116380 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-slvsr" Jul 05 13:25:42.198635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.198607 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-86bkx" Jul 05 13:25:42.203325 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.203287 2567 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-c9vhn container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.31:8080/ready\": dial tcp 10.132.0.31:8080: connect: connection refused" start-of-body= Jul 05 13:25:42.203489 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.203350 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" podUID="e933d34a-8ea4-4483-ac1f-0d0cf1b89e12" containerName="discovery" probeResult="failure" output="Get \"http://10.132.0.31:8080/ready\": dial tcp 10.132.0.31:8080: connect: connection refused" Jul 05 13:25:42.256522 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.256447 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nx7z" event={"ID":"83909d91-90dd-4e68-9104-a8d2138b50ec","Type":"ContainerStarted","Data":"0ef46f322cf8658117ff24c8f660f264463a3006c6fa740249d9871eded710ca"} Jul 05 13:25:42.509284 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.509246 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:42.539823 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.539780 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-istio-token\") pod \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " Jul 05 13:25:42.540017 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.539844 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pz297\" (UniqueName: \"kubernetes.io/projected/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-kube-api-access-pz297\") pod \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " Jul 05 13:25:42.540017 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.539937 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-local-certs\") pod \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " Jul 05 13:25:42.540017 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.539965 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-cacerts\") pod \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " Jul 05 13:25:42.540017 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.540004 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-istio-csr-dns-cert\") pod \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " Jul 05 13:25:42.540222 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.540027 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-istio-csr-ca-configmap\") pod \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " Jul 05 13:25:42.540222 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.540058 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-istio-kubeconfig\") pod \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\" (UID: \"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12\") " Jul 05 13:25:42.540731 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.540686 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "e933d34a-8ea4-4483-ac1f-0d0cf1b89e12" (UID: "e933d34a-8ea4-4483-ac1f-0d0cf1b89e12"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:25:42.542594 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.542562 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-istio-token" (OuterVolumeSpecName: "istio-token") pod "e933d34a-8ea4-4483-ac1f-0d0cf1b89e12" (UID: "e933d34a-8ea4-4483-ac1f-0d0cf1b89e12"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:25:42.542988 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.542959 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "e933d34a-8ea4-4483-ac1f-0d0cf1b89e12" (UID: "e933d34a-8ea4-4483-ac1f-0d0cf1b89e12"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:25:42.542988 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.542967 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-kube-api-access-pz297" (OuterVolumeSpecName: "kube-api-access-pz297") pod "e933d34a-8ea4-4483-ac1f-0d0cf1b89e12" (UID: "e933d34a-8ea4-4483-ac1f-0d0cf1b89e12"). InnerVolumeSpecName "kube-api-access-pz297". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:25:42.543122 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.542990 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-cacerts" (OuterVolumeSpecName: "cacerts") pod "e933d34a-8ea4-4483-ac1f-0d0cf1b89e12" (UID: "e933d34a-8ea4-4483-ac1f-0d0cf1b89e12"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:25:42.543122 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.542998 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-local-certs" (OuterVolumeSpecName: "local-certs") pod "e933d34a-8ea4-4483-ac1f-0d0cf1b89e12" (UID: "e933d34a-8ea4-4483-ac1f-0d0cf1b89e12"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:25:42.543122 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.543007 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "e933d34a-8ea4-4483-ac1f-0d0cf1b89e12" (UID: "e933d34a-8ea4-4483-ac1f-0d0cf1b89e12"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:25:42.641144 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.641055 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pz297\" (UniqueName: \"kubernetes.io/projected/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-kube-api-access-pz297\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:25:42.641144 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.641090 2567 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-local-certs\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:25:42.641144 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.641100 2567 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-cacerts\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:25:42.641144 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.641108 2567 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-istio-csr-dns-cert\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:25:42.641144 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.641118 2567 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-istio-csr-ca-configmap\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:25:42.641144 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.641127 2567 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-istio-kubeconfig\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:25:42.641144 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.641136 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12-istio-token\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:25:42.755158 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:42.755137 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-slvsr"] Jul 05 13:25:42.757190 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:25:42.757161 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod626bf575_c542_415d_9d9a_8480539083c5.slice/crio-4d19b1750d0cb7737e84840365110d2a5f2ae7c7fbd7708a894103c6a17f1d0a WatchSource:0}: Error finding container 4d19b1750d0cb7737e84840365110d2a5f2ae7c7fbd7708a894103c6a17f1d0a: Status 404 returned error can't find the container with id 4d19b1750d0cb7737e84840365110d2a5f2ae7c7fbd7708a894103c6a17f1d0a Jul 05 13:25:43.207865 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:43.207834 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-c4mkr" Jul 05 13:25:43.265937 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:43.265905 2567 generic.go:356] "Generic (PLEG): container finished" podID="e933d34a-8ea4-4483-ac1f-0d0cf1b89e12" containerID="17857b09b2458dae792d4ebaeac2f2c188a14de3270ced10a62642d1daa3895c" exitCode=0 Jul 05 13:25:43.266143 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:43.265953 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" Jul 05 13:25:43.266143 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:43.265989 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" event={"ID":"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12","Type":"ContainerDied","Data":"17857b09b2458dae792d4ebaeac2f2c188a14de3270ced10a62642d1daa3895c"} Jul 05 13:25:43.266143 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:43.266038 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn" event={"ID":"e933d34a-8ea4-4483-ac1f-0d0cf1b89e12","Type":"ContainerDied","Data":"e1f3a37298fead0a514f90602bc4d9f2a67cec2163d8c8db240137f3062dc9ce"} Jul 05 13:25:43.266143 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:43.266064 2567 scope.go:117] "RemoveContainer" containerID="17857b09b2458dae792d4ebaeac2f2c188a14de3270ced10a62642d1daa3895c" Jul 05 13:25:43.267983 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:43.267841 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-slvsr" event={"ID":"626bf575-c542-415d-9d9a-8480539083c5","Type":"ContainerStarted","Data":"4d19b1750d0cb7737e84840365110d2a5f2ae7c7fbd7708a894103c6a17f1d0a"} Jul 05 13:25:43.366222 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:43.366188 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn"] Jul 05 13:25:43.372112 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:43.372079 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-c9vhn"] Jul 05 13:25:43.396798 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:43.396773 2567 scope.go:117] "RemoveContainer" containerID="17857b09b2458dae792d4ebaeac2f2c188a14de3270ced10a62642d1daa3895c" Jul 05 13:25:43.397182 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:25:43.397163 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17857b09b2458dae792d4ebaeac2f2c188a14de3270ced10a62642d1daa3895c\": container with ID starting with 17857b09b2458dae792d4ebaeac2f2c188a14de3270ced10a62642d1daa3895c not found: ID does not exist" containerID="17857b09b2458dae792d4ebaeac2f2c188a14de3270ced10a62642d1daa3895c" Jul 05 13:25:43.397237 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:43.397194 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17857b09b2458dae792d4ebaeac2f2c188a14de3270ced10a62642d1daa3895c"} err="failed to get container status \"17857b09b2458dae792d4ebaeac2f2c188a14de3270ced10a62642d1daa3895c\": rpc error: code = NotFound desc = could not find container \"17857b09b2458dae792d4ebaeac2f2c188a14de3270ced10a62642d1daa3895c\": container with ID starting with 17857b09b2458dae792d4ebaeac2f2c188a14de3270ced10a62642d1daa3895c not found: ID does not exist" Jul 05 13:25:44.217885 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:44.217728 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e933d34a-8ea4-4483-ac1f-0d0cf1b89e12" path="/var/lib/kubelet/pods/e933d34a-8ea4-4483-ac1f-0d0cf1b89e12/volumes" Jul 05 13:25:46.287026 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:46.286984 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-slvsr" event={"ID":"626bf575-c542-415d-9d9a-8480539083c5","Type":"ContainerStarted","Data":"6890005d2065a83d8eaf482417ec28449ae4a03e8906d0383fbb2ab5ac79d3c8"} Jul 05 13:25:46.287471 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:46.287053 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-slvsr" Jul 05 13:25:46.288777 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:46.288719 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nx7z" event={"ID":"83909d91-90dd-4e68-9104-a8d2138b50ec","Type":"ContainerStarted","Data":"3e3201c3ff143569c54a5da32a8dc03fbc01d5e141047fa693abd22980bb023c"} Jul 05 13:25:46.288877 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:46.288806 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nx7z" Jul 05 13:25:46.326031 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:46.325975 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-slvsr" podStartSLOduration=2.73317187 podStartE2EDuration="5.325957801s" totalImagesPullingTime="2.592785931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:25:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:25:46.325412868 +0000 UTC m=+432.666430640" watchObservedRunningTime="2026-07-05 13:25:46.325957801 +0000 UTC m=+432.666975575" Jul 05 13:25:46.363726 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:25:46.363670 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nx7z" podStartSLOduration=1.788043271 podStartE2EDuration="5.363652898s" totalImagesPullingTime="3.575609627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:25:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:25:46.362609009 +0000 UTC m=+432.703626781" watchObservedRunningTime="2026-07-05 13:25:46.363652898 +0000 UTC m=+432.704670670" Jul 05 13:26:17.296946 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:17.296908 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7nx7z" Jul 05 13:26:17.299869 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:17.299845 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-slvsr" Jul 05 13:26:26.074037 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:26.073991 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wm7g2"] Jul 05 13:26:26.074441 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:26.074429 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e933d34a-8ea4-4483-ac1f-0d0cf1b89e12" containerName="discovery" Jul 05 13:26:26.074487 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:26.074447 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e933d34a-8ea4-4483-ac1f-0d0cf1b89e12" containerName="discovery" Jul 05 13:26:26.074527 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:26.074522 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e933d34a-8ea4-4483-ac1f-0d0cf1b89e12" containerName="discovery" Jul 05 13:26:26.076543 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:26.076526 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wm7g2" Jul 05 13:26:26.084285 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:26.084260 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-66jnl\"" Jul 05 13:26:26.084402 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:26.084298 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 05 13:26:26.084402 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:26.084325 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 05 13:26:26.107016 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:26.106961 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wm7g2"] Jul 05 13:26:26.147225 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:26.147186 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tldpz\" (UniqueName: \"kubernetes.io/projected/9781b201-c881-4c00-972e-ba0263046fbd-kube-api-access-tldpz\") pod \"kuadrant-operator-catalog-wm7g2\" (UID: \"9781b201-c881-4c00-972e-ba0263046fbd\") " pod="kuadrant-system/kuadrant-operator-catalog-wm7g2" Jul 05 13:26:26.248237 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:26.248192 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tldpz\" (UniqueName: \"kubernetes.io/projected/9781b201-c881-4c00-972e-ba0263046fbd-kube-api-access-tldpz\") pod \"kuadrant-operator-catalog-wm7g2\" (UID: \"9781b201-c881-4c00-972e-ba0263046fbd\") " pod="kuadrant-system/kuadrant-operator-catalog-wm7g2" Jul 05 13:26:26.277352 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:26.277315 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tldpz\" (UniqueName: \"kubernetes.io/projected/9781b201-c881-4c00-972e-ba0263046fbd-kube-api-access-tldpz\") pod \"kuadrant-operator-catalog-wm7g2\" (UID: \"9781b201-c881-4c00-972e-ba0263046fbd\") " pod="kuadrant-system/kuadrant-operator-catalog-wm7g2" Jul 05 13:26:26.313953 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:26.313918 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wm7g2"] Jul 05 13:26:26.314242 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:26.314226 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wm7g2" Jul 05 13:26:26.473839 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:26.473805 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wm7g2"] Jul 05 13:26:26.482539 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:26:26.482507 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9781b201_c881_4c00_972e_ba0263046fbd.slice/crio-d284cacf8b61c2476112f057e74ac8ebe74be895671692298efd79d25e7c5309 WatchSource:0}: Error finding container d284cacf8b61c2476112f057e74ac8ebe74be895671692298efd79d25e7c5309: Status 404 returned error can't find the container with id d284cacf8b61c2476112f057e74ac8ebe74be895671692298efd79d25e7c5309 Jul 05 13:26:26.489610 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:26.489577 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wm7g2" event={"ID":"9781b201-c881-4c00-972e-ba0263046fbd","Type":"ContainerStarted","Data":"d284cacf8b61c2476112f057e74ac8ebe74be895671692298efd79d25e7c5309"} Jul 05 13:26:26.541114 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:26.541071 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r8cgn"] Jul 05 13:26:26.544216 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:26.544191 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r8cgn" Jul 05 13:26:26.563124 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:26.563089 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r8cgn"] Jul 05 13:26:26.652496 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:26.652398 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7g22\" (UniqueName: \"kubernetes.io/projected/8caa73a3-2ab8-43b5-abb6-3adb2cf9e7e2-kube-api-access-r7g22\") pod \"kuadrant-operator-catalog-r8cgn\" (UID: \"8caa73a3-2ab8-43b5-abb6-3adb2cf9e7e2\") " pod="kuadrant-system/kuadrant-operator-catalog-r8cgn" Jul 05 13:26:26.753096 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:26.753053 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7g22\" (UniqueName: \"kubernetes.io/projected/8caa73a3-2ab8-43b5-abb6-3adb2cf9e7e2-kube-api-access-r7g22\") pod \"kuadrant-operator-catalog-r8cgn\" (UID: \"8caa73a3-2ab8-43b5-abb6-3adb2cf9e7e2\") " pod="kuadrant-system/kuadrant-operator-catalog-r8cgn" Jul 05 13:26:26.785304 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:26.785264 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7g22\" (UniqueName: \"kubernetes.io/projected/8caa73a3-2ab8-43b5-abb6-3adb2cf9e7e2-kube-api-access-r7g22\") pod \"kuadrant-operator-catalog-r8cgn\" (UID: \"8caa73a3-2ab8-43b5-abb6-3adb2cf9e7e2\") " pod="kuadrant-system/kuadrant-operator-catalog-r8cgn" Jul 05 13:26:26.853665 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:26.853626 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r8cgn" Jul 05 13:26:27.046531 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:27.046498 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r8cgn"] Jul 05 13:26:27.104271 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:26:27.104238 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8caa73a3_2ab8_43b5_abb6_3adb2cf9e7e2.slice/crio-d3272fa8d96054023ac80e68ce9909664b1e5cd77022be4696e4321669846389 WatchSource:0}: Error finding container d3272fa8d96054023ac80e68ce9909664b1e5cd77022be4696e4321669846389: Status 404 returned error can't find the container with id d3272fa8d96054023ac80e68ce9909664b1e5cd77022be4696e4321669846389 Jul 05 13:26:27.498356 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:27.498317 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r8cgn" event={"ID":"8caa73a3-2ab8-43b5-abb6-3adb2cf9e7e2","Type":"ContainerStarted","Data":"d3272fa8d96054023ac80e68ce9909664b1e5cd77022be4696e4321669846389"} Jul 05 13:26:28.507497 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:28.507395 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-wm7g2" podUID="9781b201-c881-4c00-972e-ba0263046fbd" containerName="registry-server" containerID="cri-o://a8d291346b4eacc87e8c9faa9c778064e5361879b1c9d1e0ecd60605916e245f" gracePeriod=2 Jul 05 13:26:28.507497 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:28.507450 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wm7g2" event={"ID":"9781b201-c881-4c00-972e-ba0263046fbd","Type":"ContainerStarted","Data":"a8d291346b4eacc87e8c9faa9c778064e5361879b1c9d1e0ecd60605916e245f"} Jul 05 13:26:28.509672 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:28.509646 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r8cgn" event={"ID":"8caa73a3-2ab8-43b5-abb6-3adb2cf9e7e2","Type":"ContainerStarted","Data":"f05fe1f709d654efc804f9826bb887a6b43c6b7b7c43954478f3dee3101e0ddb"} Jul 05 13:26:28.553576 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:28.553522 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-wm7g2" podStartSLOduration=0.688477955 podStartE2EDuration="2.553503962s" totalImagesPullingTime="1.865026007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:26:28.553187906 +0000 UTC m=+474.894205678" watchObservedRunningTime="2026-07-05 13:26:28.553503962 +0000 UTC m=+474.894521737" Jul 05 13:26:28.587239 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:28.587179 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-r8cgn" podStartSLOduration=1.343273934 podStartE2EDuration="2.587153586s" totalImagesPullingTime="1.243879652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:26:28.584300842 +0000 UTC m=+474.925318614" watchObservedRunningTime="2026-07-05 13:26:28.587153586 +0000 UTC m=+474.928171358" Jul 05 13:26:28.762900 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:28.762874 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wm7g2" Jul 05 13:26:28.874655 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:28.874557 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tldpz\" (UniqueName: \"kubernetes.io/projected/9781b201-c881-4c00-972e-ba0263046fbd-kube-api-access-tldpz\") pod \"9781b201-c881-4c00-972e-ba0263046fbd\" (UID: \"9781b201-c881-4c00-972e-ba0263046fbd\") " Jul 05 13:26:28.876882 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:28.876856 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9781b201-c881-4c00-972e-ba0263046fbd-kube-api-access-tldpz" (OuterVolumeSpecName: "kube-api-access-tldpz") pod "9781b201-c881-4c00-972e-ba0263046fbd" (UID: "9781b201-c881-4c00-972e-ba0263046fbd"). InnerVolumeSpecName "kube-api-access-tldpz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:26:28.976095 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:28.976053 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tldpz\" (UniqueName: \"kubernetes.io/projected/9781b201-c881-4c00-972e-ba0263046fbd-kube-api-access-tldpz\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:26:29.515491 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:29.515453 2567 generic.go:356] "Generic (PLEG): container finished" podID="9781b201-c881-4c00-972e-ba0263046fbd" containerID="a8d291346b4eacc87e8c9faa9c778064e5361879b1c9d1e0ecd60605916e245f" exitCode=0 Jul 05 13:26:29.515491 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:29.515494 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wm7g2" Jul 05 13:26:29.516019 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:29.515538 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wm7g2" event={"ID":"9781b201-c881-4c00-972e-ba0263046fbd","Type":"ContainerDied","Data":"a8d291346b4eacc87e8c9faa9c778064e5361879b1c9d1e0ecd60605916e245f"} Jul 05 13:26:29.516019 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:29.515595 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wm7g2" event={"ID":"9781b201-c881-4c00-972e-ba0263046fbd","Type":"ContainerDied","Data":"d284cacf8b61c2476112f057e74ac8ebe74be895671692298efd79d25e7c5309"} Jul 05 13:26:29.516019 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:29.515618 2567 scope.go:117] "RemoveContainer" containerID="a8d291346b4eacc87e8c9faa9c778064e5361879b1c9d1e0ecd60605916e245f" Jul 05 13:26:29.525163 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:29.525144 2567 scope.go:117] "RemoveContainer" containerID="a8d291346b4eacc87e8c9faa9c778064e5361879b1c9d1e0ecd60605916e245f" Jul 05 13:26:29.525458 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:26:29.525437 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8d291346b4eacc87e8c9faa9c778064e5361879b1c9d1e0ecd60605916e245f\": container with ID starting with a8d291346b4eacc87e8c9faa9c778064e5361879b1c9d1e0ecd60605916e245f not found: ID does not exist" containerID="a8d291346b4eacc87e8c9faa9c778064e5361879b1c9d1e0ecd60605916e245f" Jul 05 13:26:29.525550 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:29.525467 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8d291346b4eacc87e8c9faa9c778064e5361879b1c9d1e0ecd60605916e245f"} err="failed to get container status \"a8d291346b4eacc87e8c9faa9c778064e5361879b1c9d1e0ecd60605916e245f\": rpc error: code = NotFound desc = could not find container \"a8d291346b4eacc87e8c9faa9c778064e5361879b1c9d1e0ecd60605916e245f\": container with ID starting with a8d291346b4eacc87e8c9faa9c778064e5361879b1c9d1e0ecd60605916e245f not found: ID does not exist" Jul 05 13:26:29.547610 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:29.547570 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wm7g2"] Jul 05 13:26:29.552146 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:29.552113 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wm7g2"] Jul 05 13:26:30.216522 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:30.216492 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9781b201-c881-4c00-972e-ba0263046fbd" path="/var/lib/kubelet/pods/9781b201-c881-4c00-972e-ba0263046fbd/volumes" Jul 05 13:26:36.854352 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:36.854305 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-r8cgn" Jul 05 13:26:36.854797 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:36.854400 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-r8cgn" Jul 05 13:26:36.878003 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:36.877967 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-r8cgn" Jul 05 13:26:37.579078 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:37.579050 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-r8cgn" Jul 05 13:26:55.387218 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:55.387185 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t"] Jul 05 13:26:55.387697 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:55.387576 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9781b201-c881-4c00-972e-ba0263046fbd" containerName="registry-server" Jul 05 13:26:55.387697 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:55.387587 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9781b201-c881-4c00-972e-ba0263046fbd" containerName="registry-server" Jul 05 13:26:55.387697 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:55.387666 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9781b201-c881-4c00-972e-ba0263046fbd" containerName="registry-server" Jul 05 13:26:55.392432 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:55.392411 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" Jul 05 13:26:55.404122 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:55.404092 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-5tssm\"" Jul 05 13:26:55.410888 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:55.410856 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t"] Jul 05 13:26:55.524387 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:55.524344 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f47195cc-93db-4a69-89c3-1350e929c377-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2292t\" (UID: \"f47195cc-93db-4a69-89c3-1350e929c377\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" Jul 05 13:26:55.524589 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:55.524407 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gfqr\" (UniqueName: \"kubernetes.io/projected/f47195cc-93db-4a69-89c3-1350e929c377-kube-api-access-9gfqr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2292t\" (UID: \"f47195cc-93db-4a69-89c3-1350e929c377\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" Jul 05 13:26:55.625890 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:55.625847 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f47195cc-93db-4a69-89c3-1350e929c377-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2292t\" (UID: \"f47195cc-93db-4a69-89c3-1350e929c377\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" Jul 05 13:26:55.626091 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:55.625912 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gfqr\" (UniqueName: \"kubernetes.io/projected/f47195cc-93db-4a69-89c3-1350e929c377-kube-api-access-9gfqr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2292t\" (UID: \"f47195cc-93db-4a69-89c3-1350e929c377\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" Jul 05 13:26:55.626249 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:55.626227 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f47195cc-93db-4a69-89c3-1350e929c377-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2292t\" (UID: \"f47195cc-93db-4a69-89c3-1350e929c377\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" Jul 05 13:26:55.647400 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:55.647319 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gfqr\" (UniqueName: \"kubernetes.io/projected/f47195cc-93db-4a69-89c3-1350e929c377-kube-api-access-9gfqr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2292t\" (UID: \"f47195cc-93db-4a69-89c3-1350e929c377\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" Jul 05 13:26:55.706038 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:55.705978 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" Jul 05 13:26:55.887086 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:55.887046 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t"] Jul 05 13:26:55.901334 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:26:55.901300 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf47195cc_93db_4a69_89c3_1350e929c377.slice/crio-a311a66dd961b81550ab2fac578a09218505bee18f4710d6d0dea6b6296a031c WatchSource:0}: Error finding container a311a66dd961b81550ab2fac578a09218505bee18f4710d6d0dea6b6296a031c: Status 404 returned error can't find the container with id a311a66dd961b81550ab2fac578a09218505bee18f4710d6d0dea6b6296a031c Jul 05 13:26:56.651496 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:56.651462 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" event={"ID":"f47195cc-93db-4a69-89c3-1350e929c377","Type":"ContainerStarted","Data":"a311a66dd961b81550ab2fac578a09218505bee18f4710d6d0dea6b6296a031c"} Jul 05 13:26:57.458418 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:57.458383 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-pzwwt"] Jul 05 13:26:57.464276 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:57.464247 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pzwwt" Jul 05 13:26:57.468257 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:57.468230 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 05 13:26:57.468429 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:57.468296 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-pg86s\"" Jul 05 13:26:57.477047 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:57.477014 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-pzwwt"] Jul 05 13:26:57.644124 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:57.644079 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trvz9\" (UniqueName: \"kubernetes.io/projected/73e4e360-0c3b-4bdf-82e9-b6ce19235329-kube-api-access-trvz9\") pod \"dns-operator-controller-manager-864dd78c4c-pzwwt\" (UID: \"73e4e360-0c3b-4bdf-82e9-b6ce19235329\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pzwwt" Jul 05 13:26:57.744764 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:57.744652 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trvz9\" (UniqueName: \"kubernetes.io/projected/73e4e360-0c3b-4bdf-82e9-b6ce19235329-kube-api-access-trvz9\") pod \"dns-operator-controller-manager-864dd78c4c-pzwwt\" (UID: \"73e4e360-0c3b-4bdf-82e9-b6ce19235329\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pzwwt" Jul 05 13:26:57.755410 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:57.755382 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trvz9\" (UniqueName: \"kubernetes.io/projected/73e4e360-0c3b-4bdf-82e9-b6ce19235329-kube-api-access-trvz9\") pod \"dns-operator-controller-manager-864dd78c4c-pzwwt\" (UID: \"73e4e360-0c3b-4bdf-82e9-b6ce19235329\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pzwwt" Jul 05 13:26:57.777210 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:57.777170 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pzwwt" Jul 05 13:26:57.958717 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:57.958636 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-pzwwt"] Jul 05 13:26:57.962951 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:26:57.962914 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73e4e360_0c3b_4bdf_82e9_b6ce19235329.slice/crio-e79101fda63770b504833d984fac0949d005b05d5bfeb3cea5c3c02833b40d84 WatchSource:0}: Error finding container e79101fda63770b504833d984fac0949d005b05d5bfeb3cea5c3c02833b40d84: Status 404 returned error can't find the container with id e79101fda63770b504833d984fac0949d005b05d5bfeb3cea5c3c02833b40d84 Jul 05 13:26:58.664347 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:58.664308 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pzwwt" event={"ID":"73e4e360-0c3b-4bdf-82e9-b6ce19235329","Type":"ContainerStarted","Data":"e79101fda63770b504833d984fac0949d005b05d5bfeb3cea5c3c02833b40d84"} Jul 05 13:26:59.672866 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:59.672830 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" event={"ID":"f47195cc-93db-4a69-89c3-1350e929c377","Type":"ContainerStarted","Data":"43ff258b2c0b61ade219bb576824275ecf4988ab2229c8811d5cd0a42976e364"} Jul 05 13:26:59.673248 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:26:59.672924 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" Jul 05 13:27:00.709990 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:00.709921 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" podStartSLOduration=2.040465229 podStartE2EDuration="5.709898615s" totalImagesPullingTime="3.669433386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:26:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:26:59.700627409 +0000 UTC m=+506.041645174" watchObservedRunningTime="2026-07-05 13:27:00.709898615 +0000 UTC m=+507.050916388" Jul 05 13:27:00.712115 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:00.712091 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd"] Jul 05 13:27:00.716281 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:00.716255 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd" Jul 05 13:27:00.725576 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:00.725553 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-vnpqj\"" Jul 05 13:27:00.730508 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:00.730484 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd"] Jul 05 13:27:00.876965 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:00.876921 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm2fn\" (UniqueName: \"kubernetes.io/projected/adce2775-2e72-4a05-883c-f5f80efa30ba-kube-api-access-nm2fn\") pod \"limitador-operator-controller-manager-859d7b78bd-hflvd\" (UID: \"adce2775-2e72-4a05-883c-f5f80efa30ba\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd" Jul 05 13:27:00.977817 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:00.977674 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm2fn\" (UniqueName: \"kubernetes.io/projected/adce2775-2e72-4a05-883c-f5f80efa30ba-kube-api-access-nm2fn\") pod \"limitador-operator-controller-manager-859d7b78bd-hflvd\" (UID: \"adce2775-2e72-4a05-883c-f5f80efa30ba\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd" Jul 05 13:27:00.991677 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:00.991638 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm2fn\" (UniqueName: \"kubernetes.io/projected/adce2775-2e72-4a05-883c-f5f80efa30ba-kube-api-access-nm2fn\") pod \"limitador-operator-controller-manager-859d7b78bd-hflvd\" (UID: \"adce2775-2e72-4a05-883c-f5f80efa30ba\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd" Jul 05 13:27:01.028441 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:01.028404 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd" Jul 05 13:27:01.193429 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:01.193400 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd"] Jul 05 13:27:01.195381 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:27:01.195353 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadce2775_2e72_4a05_883c_f5f80efa30ba.slice/crio-687e86e07683a7ba3d4c7b127303c2a487ec474e30f31c9ffad5632efd3c68c3 WatchSource:0}: Error finding container 687e86e07683a7ba3d4c7b127303c2a487ec474e30f31c9ffad5632efd3c68c3: Status 404 returned error can't find the container with id 687e86e07683a7ba3d4c7b127303c2a487ec474e30f31c9ffad5632efd3c68c3 Jul 05 13:27:01.690527 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:01.690490 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pzwwt" event={"ID":"73e4e360-0c3b-4bdf-82e9-b6ce19235329","Type":"ContainerStarted","Data":"9f5c058b5d54cea611601fc4ac5e2215d6145539ac4298cb4681cedcdc08c820"} Jul 05 13:27:01.690728 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:01.690598 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pzwwt" Jul 05 13:27:01.693163 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:01.693130 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd" event={"ID":"adce2775-2e72-4a05-883c-f5f80efa30ba","Type":"ContainerStarted","Data":"687e86e07683a7ba3d4c7b127303c2a487ec474e30f31c9ffad5632efd3c68c3"} Jul 05 13:27:01.735385 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:01.735324 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pzwwt" podStartSLOduration=1.821852609 podStartE2EDuration="4.73530159s" totalImagesPullingTime="2.913448981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:27:01.734221338 +0000 UTC m=+508.075239111" watchObservedRunningTime="2026-07-05 13:27:01.73530159 +0000 UTC m=+508.076319363" Jul 05 13:27:02.700086 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:02.699990 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd" event={"ID":"adce2775-2e72-4a05-883c-f5f80efa30ba","Type":"ContainerStarted","Data":"4b38acf668968a32e664a708bd7e2373e85afd02c52c1a9564c3fa7d5335e2b0"} Jul 05 13:27:03.704918 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:03.704891 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd" Jul 05 13:27:04.319253 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:04.319197 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd" podStartSLOduration=3.087045816 podStartE2EDuration="4.319176247s" totalImagesPullingTime="1.232130431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:27:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:27:02.741488921 +0000 UTC m=+509.082506692" watchObservedRunningTime="2026-07-05 13:27:04.319176247 +0000 UTC m=+510.660194057" Jul 05 13:27:04.320413 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:04.320394 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-zpsjv"] Jul 05 13:27:04.324261 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:04.324241 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zpsjv" Jul 05 13:27:04.330441 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:04.330420 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-ts84h\"" Jul 05 13:27:04.357406 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:04.357374 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-zpsjv"] Jul 05 13:27:04.407673 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:04.407634 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snqwn\" (UniqueName: \"kubernetes.io/projected/132eb69a-8cde-4a80-a386-7b101e2285ff-kube-api-access-snqwn\") pod \"authorino-operator-75cc8d5cb7-zpsjv\" (UID: \"132eb69a-8cde-4a80-a386-7b101e2285ff\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-zpsjv" Jul 05 13:27:04.509283 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:04.509246 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snqwn\" (UniqueName: \"kubernetes.io/projected/132eb69a-8cde-4a80-a386-7b101e2285ff-kube-api-access-snqwn\") pod \"authorino-operator-75cc8d5cb7-zpsjv\" (UID: \"132eb69a-8cde-4a80-a386-7b101e2285ff\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-zpsjv" Jul 05 13:27:04.521836 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:04.521805 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snqwn\" (UniqueName: \"kubernetes.io/projected/132eb69a-8cde-4a80-a386-7b101e2285ff-kube-api-access-snqwn\") pod \"authorino-operator-75cc8d5cb7-zpsjv\" (UID: \"132eb69a-8cde-4a80-a386-7b101e2285ff\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-zpsjv" Jul 05 13:27:04.635091 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:04.634997 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zpsjv" Jul 05 13:27:04.798751 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:04.798706 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-zpsjv"] Jul 05 13:27:04.801066 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:27:04.801039 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod132eb69a_8cde_4a80_a386_7b101e2285ff.slice/crio-f6f4aa918bcfa7455186acf95739dc40d9c980e3b2168190ee3984e9b13e3a8d WatchSource:0}: Error finding container f6f4aa918bcfa7455186acf95739dc40d9c980e3b2168190ee3984e9b13e3a8d: Status 404 returned error can't find the container with id f6f4aa918bcfa7455186acf95739dc40d9c980e3b2168190ee3984e9b13e3a8d Jul 05 13:27:05.719384 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:05.719341 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zpsjv" event={"ID":"132eb69a-8cde-4a80-a386-7b101e2285ff","Type":"ContainerStarted","Data":"f6f4aa918bcfa7455186acf95739dc40d9c980e3b2168190ee3984e9b13e3a8d"} Jul 05 13:27:07.732380 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:07.732344 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zpsjv" event={"ID":"132eb69a-8cde-4a80-a386-7b101e2285ff","Type":"ContainerStarted","Data":"cc72dd7c3d49535e5f1d454ae45c5c2ab5d7b7175bcb0d062f47ef8a42735dc3"} Jul 05 13:27:07.732779 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:07.732408 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zpsjv" Jul 05 13:27:07.761059 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:07.761006 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zpsjv" podStartSLOduration=1.6698755859999999 podStartE2EDuration="3.760987632s" totalImagesPullingTime="2.091112046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:27:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:27:07.759920286 +0000 UTC m=+514.100938058" watchObservedRunningTime="2026-07-05 13:27:07.760987632 +0000 UTC m=+514.102005404" Jul 05 13:27:10.681835 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:10.681791 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" Jul 05 13:27:12.338870 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.338816 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t"] Jul 05 13:27:12.339370 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.339112 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" podUID="f47195cc-93db-4a69-89c3-1350e929c377" containerName="manager" containerID="cri-o://43ff258b2c0b61ade219bb576824275ecf4988ab2229c8811d5cd0a42976e364" gracePeriod=2 Jul 05 13:27:12.342201 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:12.342161 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2292t\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="f47195cc-93db-4a69-89c3-1350e929c377" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" Jul 05 13:27:12.342321 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.342277 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t"] Jul 05 13:27:12.364665 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.364223 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd"] Jul 05 13:27:12.364665 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.364473 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd" podUID="adce2775-2e72-4a05-883c-f5f80efa30ba" containerName="manager" containerID="cri-o://4b38acf668968a32e664a708bd7e2373e85afd02c52c1a9564c3fa7d5335e2b0" gracePeriod=2 Jul 05 13:27:12.366573 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.366541 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd" Jul 05 13:27:12.367308 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.367284 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sp7mv"] Jul 05 13:27:12.367435 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:12.367399 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2292t\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="f47195cc-93db-4a69-89c3-1350e929c377" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" Jul 05 13:27:12.367960 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.367938 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f47195cc-93db-4a69-89c3-1350e929c377" containerName="manager" Jul 05 13:27:12.368059 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.367963 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f47195cc-93db-4a69-89c3-1350e929c377" containerName="manager" Jul 05 13:27:12.368132 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.368087 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f47195cc-93db-4a69-89c3-1350e929c377" containerName="manager" Jul 05 13:27:12.371953 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:12.371905 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hflvd\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="adce2775-2e72-4a05-883c-f5f80efa30ba" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd" Jul 05 13:27:12.372521 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.372499 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd"] Jul 05 13:27:12.372634 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.372617 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sp7mv" Jul 05 13:27:12.374532 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:12.374499 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hflvd\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="adce2775-2e72-4a05-883c-f5f80efa30ba" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd" Jul 05 13:27:12.379578 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:12.379541 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2292t\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="f47195cc-93db-4a69-89c3-1350e929c377" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" Jul 05 13:27:12.382090 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:12.382063 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2292t\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="f47195cc-93db-4a69-89c3-1350e929c377" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" Jul 05 13:27:12.399050 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.399017 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sp7mv"] Jul 05 13:27:12.416284 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.416234 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9rcph"] Jul 05 13:27:12.417828 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.417177 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adce2775-2e72-4a05-883c-f5f80efa30ba" containerName="manager" Jul 05 13:27:12.417828 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.417201 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="adce2775-2e72-4a05-883c-f5f80efa30ba" containerName="manager" Jul 05 13:27:12.417828 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:12.417293 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hflvd\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="adce2775-2e72-4a05-883c-f5f80efa30ba" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd" Jul 05 13:27:12.417828 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.417331 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="adce2775-2e72-4a05-883c-f5f80efa30ba" containerName="manager" Jul 05 13:27:12.420513 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.420492 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9rcph" Jul 05 13:27:12.423043 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:12.423021 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2292t\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="f47195cc-93db-4a69-89c3-1350e929c377" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" Jul 05 13:27:12.430990 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.430963 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9rcph"] Jul 05 13:27:12.449588 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:12.449543 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hflvd\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="adce2775-2e72-4a05-883c-f5f80efa30ba" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd" Jul 05 13:27:12.489372 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.489343 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l67zq\" (UniqueName: \"kubernetes.io/projected/c43e9147-0d13-403b-8e70-15aebd8de13a-kube-api-access-l67zq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sp7mv\" (UID: \"c43e9147-0d13-403b-8e70-15aebd8de13a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sp7mv" Jul 05 13:27:12.489519 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.489387 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c43e9147-0d13-403b-8e70-15aebd8de13a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sp7mv\" (UID: \"c43e9147-0d13-403b-8e70-15aebd8de13a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sp7mv" Jul 05 13:27:12.489519 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.489429 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqbwd\" (UniqueName: \"kubernetes.io/projected/36c457af-8be7-4661-8eec-b4d158f81993-kube-api-access-bqbwd\") pod \"limitador-operator-controller-manager-859d7b78bd-9rcph\" (UID: \"36c457af-8be7-4661-8eec-b4d158f81993\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9rcph" Jul 05 13:27:12.590649 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.590555 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqbwd\" (UniqueName: \"kubernetes.io/projected/36c457af-8be7-4661-8eec-b4d158f81993-kube-api-access-bqbwd\") pod \"limitador-operator-controller-manager-859d7b78bd-9rcph\" (UID: \"36c457af-8be7-4661-8eec-b4d158f81993\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9rcph" Jul 05 13:27:12.590830 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.590712 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l67zq\" (UniqueName: \"kubernetes.io/projected/c43e9147-0d13-403b-8e70-15aebd8de13a-kube-api-access-l67zq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sp7mv\" (UID: \"c43e9147-0d13-403b-8e70-15aebd8de13a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sp7mv" Jul 05 13:27:12.590830 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.590778 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c43e9147-0d13-403b-8e70-15aebd8de13a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sp7mv\" (UID: \"c43e9147-0d13-403b-8e70-15aebd8de13a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sp7mv" Jul 05 13:27:12.591163 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.591138 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c43e9147-0d13-403b-8e70-15aebd8de13a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sp7mv\" (UID: \"c43e9147-0d13-403b-8e70-15aebd8de13a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sp7mv" Jul 05 13:27:12.599216 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.599182 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l67zq\" (UniqueName: \"kubernetes.io/projected/c43e9147-0d13-403b-8e70-15aebd8de13a-kube-api-access-l67zq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sp7mv\" (UID: \"c43e9147-0d13-403b-8e70-15aebd8de13a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sp7mv" Jul 05 13:27:12.599416 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.599397 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqbwd\" (UniqueName: \"kubernetes.io/projected/36c457af-8be7-4661-8eec-b4d158f81993-kube-api-access-bqbwd\") pod \"limitador-operator-controller-manager-859d7b78bd-9rcph\" (UID: \"36c457af-8be7-4661-8eec-b4d158f81993\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9rcph" Jul 05 13:27:12.623974 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.623950 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" Jul 05 13:27:12.625921 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.625903 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd" Jul 05 13:27:12.626471 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:12.626449 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hflvd\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="adce2775-2e72-4a05-883c-f5f80efa30ba" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd" Jul 05 13:27:12.628642 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:12.628618 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2292t\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="f47195cc-93db-4a69-89c3-1350e929c377" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" Jul 05 13:27:12.630810 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:12.630791 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2292t\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="f47195cc-93db-4a69-89c3-1350e929c377" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" Jul 05 13:27:12.632838 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:12.632815 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hflvd\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="adce2775-2e72-4a05-883c-f5f80efa30ba" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd" Jul 05 13:27:12.691686 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.691647 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gfqr\" (UniqueName: \"kubernetes.io/projected/f47195cc-93db-4a69-89c3-1350e929c377-kube-api-access-9gfqr\") pod \"f47195cc-93db-4a69-89c3-1350e929c377\" (UID: \"f47195cc-93db-4a69-89c3-1350e929c377\") " Jul 05 13:27:12.691929 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.691715 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nm2fn\" (UniqueName: \"kubernetes.io/projected/adce2775-2e72-4a05-883c-f5f80efa30ba-kube-api-access-nm2fn\") pod \"adce2775-2e72-4a05-883c-f5f80efa30ba\" (UID: \"adce2775-2e72-4a05-883c-f5f80efa30ba\") " Jul 05 13:27:12.691929 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.691781 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f47195cc-93db-4a69-89c3-1350e929c377-extensions-socket-volume\") pod \"f47195cc-93db-4a69-89c3-1350e929c377\" (UID: \"f47195cc-93db-4a69-89c3-1350e929c377\") " Jul 05 13:27:12.692356 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.692322 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f47195cc-93db-4a69-89c3-1350e929c377-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f47195cc-93db-4a69-89c3-1350e929c377" (UID: "f47195cc-93db-4a69-89c3-1350e929c377"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:27:12.694054 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.694023 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f47195cc-93db-4a69-89c3-1350e929c377-kube-api-access-9gfqr" (OuterVolumeSpecName: "kube-api-access-9gfqr") pod "f47195cc-93db-4a69-89c3-1350e929c377" (UID: "f47195cc-93db-4a69-89c3-1350e929c377"). InnerVolumeSpecName "kube-api-access-9gfqr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:27:12.694143 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.694070 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adce2775-2e72-4a05-883c-f5f80efa30ba-kube-api-access-nm2fn" (OuterVolumeSpecName: "kube-api-access-nm2fn") pod "adce2775-2e72-4a05-883c-f5f80efa30ba" (UID: "adce2775-2e72-4a05-883c-f5f80efa30ba"). InnerVolumeSpecName "kube-api-access-nm2fn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:27:12.702311 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.702289 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pzwwt" Jul 05 13:27:12.705204 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:12.705178 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2292t\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="f47195cc-93db-4a69-89c3-1350e929c377" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" Jul 05 13:27:12.725729 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.725701 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t4bhg"] Jul 05 13:27:12.729468 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.729448 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t4bhg" Jul 05 13:27:12.751561 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:12.751517 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hflvd\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="adce2775-2e72-4a05-883c-f5f80efa30ba" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd" Jul 05 13:27:12.752298 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.752280 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t4bhg"] Jul 05 13:27:12.753900 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:12.753871 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2292t\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="f47195cc-93db-4a69-89c3-1350e929c377" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" Jul 05 13:27:12.756477 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:12.756443 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hflvd\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="adce2775-2e72-4a05-883c-f5f80efa30ba" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd" Jul 05 13:27:12.758985 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.758962 2567 generic.go:356] "Generic (PLEG): container finished" podID="f47195cc-93db-4a69-89c3-1350e929c377" containerID="43ff258b2c0b61ade219bb576824275ecf4988ab2229c8811d5cd0a42976e364" exitCode=0 Jul 05 13:27:12.759078 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.759005 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" Jul 05 13:27:12.759078 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.759064 2567 scope.go:117] "RemoveContainer" containerID="43ff258b2c0b61ade219bb576824275ecf4988ab2229c8811d5cd0a42976e364" Jul 05 13:27:12.760772 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.760735 2567 generic.go:356] "Generic (PLEG): container finished" podID="adce2775-2e72-4a05-883c-f5f80efa30ba" containerID="4b38acf668968a32e664a708bd7e2373e85afd02c52c1a9564c3fa7d5335e2b0" exitCode=0 Jul 05 13:27:12.760883 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.760867 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd" Jul 05 13:27:12.771941 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.771827 2567 scope.go:117] "RemoveContainer" containerID="43ff258b2c0b61ade219bb576824275ecf4988ab2229c8811d5cd0a42976e364" Jul 05 13:27:12.772261 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:12.772203 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43ff258b2c0b61ade219bb576824275ecf4988ab2229c8811d5cd0a42976e364\": container with ID starting with 43ff258b2c0b61ade219bb576824275ecf4988ab2229c8811d5cd0a42976e364 not found: ID does not exist" containerID="43ff258b2c0b61ade219bb576824275ecf4988ab2229c8811d5cd0a42976e364" Jul 05 13:27:12.772538 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.772414 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43ff258b2c0b61ade219bb576824275ecf4988ab2229c8811d5cd0a42976e364"} err="failed to get container status \"43ff258b2c0b61ade219bb576824275ecf4988ab2229c8811d5cd0a42976e364\": rpc error: code = NotFound desc = could not find container \"43ff258b2c0b61ade219bb576824275ecf4988ab2229c8811d5cd0a42976e364\": container with ID starting with 43ff258b2c0b61ade219bb576824275ecf4988ab2229c8811d5cd0a42976e364 not found: ID does not exist" Jul 05 13:27:12.772538 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.772447 2567 scope.go:117] "RemoveContainer" containerID="4b38acf668968a32e664a708bd7e2373e85afd02c52c1a9564c3fa7d5335e2b0" Jul 05 13:27:12.781700 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.781675 2567 scope.go:117] "RemoveContainer" containerID="4b38acf668968a32e664a708bd7e2373e85afd02c52c1a9564c3fa7d5335e2b0" Jul 05 13:27:12.782053 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:12.782029 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b38acf668968a32e664a708bd7e2373e85afd02c52c1a9564c3fa7d5335e2b0\": container with ID starting with 4b38acf668968a32e664a708bd7e2373e85afd02c52c1a9564c3fa7d5335e2b0 not found: ID does not exist" containerID="4b38acf668968a32e664a708bd7e2373e85afd02c52c1a9564c3fa7d5335e2b0" Jul 05 13:27:12.782131 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.782067 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b38acf668968a32e664a708bd7e2373e85afd02c52c1a9564c3fa7d5335e2b0"} err="failed to get container status \"4b38acf668968a32e664a708bd7e2373e85afd02c52c1a9564c3fa7d5335e2b0\": rpc error: code = NotFound desc = could not find container \"4b38acf668968a32e664a708bd7e2373e85afd02c52c1a9564c3fa7d5335e2b0\": container with ID starting with 4b38acf668968a32e664a708bd7e2373e85afd02c52c1a9564c3fa7d5335e2b0 not found: ID does not exist" Jul 05 13:27:12.791709 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:12.791668 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2292t\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="f47195cc-93db-4a69-89c3-1350e929c377" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" Jul 05 13:27:12.791895 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.791688 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sp7mv" Jul 05 13:27:12.793067 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.793033 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgl79\" (UniqueName: \"kubernetes.io/projected/b2d46bf8-5c47-40bc-89e5-62d9ac618410-kube-api-access-pgl79\") pod \"kuadrant-operator-controller-manager-86878c769f-t4bhg\" (UID: \"b2d46bf8-5c47-40bc-89e5-62d9ac618410\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t4bhg" Jul 05 13:27:12.793239 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.793217 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2d46bf8-5c47-40bc-89e5-62d9ac618410-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-t4bhg\" (UID: \"b2d46bf8-5c47-40bc-89e5-62d9ac618410\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t4bhg" Jul 05 13:27:12.793367 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.793338 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gfqr\" (UniqueName: \"kubernetes.io/projected/f47195cc-93db-4a69-89c3-1350e929c377-kube-api-access-9gfqr\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:27:12.793367 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.793362 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nm2fn\" (UniqueName: \"kubernetes.io/projected/adce2775-2e72-4a05-883c-f5f80efa30ba-kube-api-access-nm2fn\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:27:12.793530 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.793377 2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f47195cc-93db-4a69-89c3-1350e929c377-extensions-socket-volume\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:27:12.796280 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:12.796259 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hflvd\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="adce2775-2e72-4a05-883c-f5f80efa30ba" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd" Jul 05 13:27:12.800435 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.800411 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9rcph" Jul 05 13:27:12.894567 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.894527 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2d46bf8-5c47-40bc-89e5-62d9ac618410-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-t4bhg\" (UID: \"b2d46bf8-5c47-40bc-89e5-62d9ac618410\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t4bhg" Jul 05 13:27:12.894725 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.894620 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgl79\" (UniqueName: \"kubernetes.io/projected/b2d46bf8-5c47-40bc-89e5-62d9ac618410-kube-api-access-pgl79\") pod \"kuadrant-operator-controller-manager-86878c769f-t4bhg\" (UID: \"b2d46bf8-5c47-40bc-89e5-62d9ac618410\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t4bhg" Jul 05 13:27:12.895478 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.895452 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2d46bf8-5c47-40bc-89e5-62d9ac618410-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-t4bhg\" (UID: \"b2d46bf8-5c47-40bc-89e5-62d9ac618410\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t4bhg" Jul 05 13:27:12.904641 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.904610 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgl79\" (UniqueName: \"kubernetes.io/projected/b2d46bf8-5c47-40bc-89e5-62d9ac618410-kube-api-access-pgl79\") pod \"kuadrant-operator-controller-manager-86878c769f-t4bhg\" (UID: \"b2d46bf8-5c47-40bc-89e5-62d9ac618410\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t4bhg" Jul 05 13:27:12.960001 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.959926 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sp7mv"] Jul 05 13:27:12.962576 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:27:12.962549 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc43e9147_0d13_403b_8e70_15aebd8de13a.slice/crio-7e5f38b97759a073148911061514f7821ff92f4af11ff83c6476e7348fb1e231 WatchSource:0}: Error finding container 7e5f38b97759a073148911061514f7821ff92f4af11ff83c6476e7348fb1e231: Status 404 returned error can't find the container with id 7e5f38b97759a073148911061514f7821ff92f4af11ff83c6476e7348fb1e231 Jul 05 13:27:12.982620 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:12.982593 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9rcph"] Jul 05 13:27:12.984082 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:27:12.984038 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36c457af_8be7_4661_8eec_b4d158f81993.slice/crio-6314a71996e1bcac8d343041791438471bcc09786ae70b9676d1a5681ff4ede9 WatchSource:0}: Error finding container 6314a71996e1bcac8d343041791438471bcc09786ae70b9676d1a5681ff4ede9: Status 404 returned error can't find the container with id 6314a71996e1bcac8d343041791438471bcc09786ae70b9676d1a5681ff4ede9 Jul 05 13:27:13.039865 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:13.039842 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t4bhg" Jul 05 13:27:13.194481 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:13.194439 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t4bhg"] Jul 05 13:27:13.196460 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:27:13.196434 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2d46bf8_5c47_40bc_89e5_62d9ac618410.slice/crio-f9af8dfe5d65a3d7aa629adab1c1a20bde20d55742b17b58a9d306a3288f9546 WatchSource:0}: Error finding container f9af8dfe5d65a3d7aa629adab1c1a20bde20d55742b17b58a9d306a3288f9546: Status 404 returned error can't find the container with id f9af8dfe5d65a3d7aa629adab1c1a20bde20d55742b17b58a9d306a3288f9546 Jul 05 13:27:13.767936 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:13.767890 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t4bhg" event={"ID":"b2d46bf8-5c47-40bc-89e5-62d9ac618410","Type":"ContainerStarted","Data":"8356cbcc74cae9affac643c7e8a95aabf92641c483431ac7470439c2a03c7a59"} Jul 05 13:27:13.768415 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:13.767999 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t4bhg" event={"ID":"b2d46bf8-5c47-40bc-89e5-62d9ac618410","Type":"ContainerStarted","Data":"f9af8dfe5d65a3d7aa629adab1c1a20bde20d55742b17b58a9d306a3288f9546"} Jul 05 13:27:13.768415 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:13.768033 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t4bhg" Jul 05 13:27:13.769520 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:13.769489 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9rcph" event={"ID":"36c457af-8be7-4661-8eec-b4d158f81993","Type":"ContainerStarted","Data":"9155c41704a8b1f36ce6f4cd4eff23a55e18b55bb6c0f0e07908bc7ed0880749"} Jul 05 13:27:13.769657 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:13.769546 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9rcph" event={"ID":"36c457af-8be7-4661-8eec-b4d158f81993","Type":"ContainerStarted","Data":"6314a71996e1bcac8d343041791438471bcc09786ae70b9676d1a5681ff4ede9"} Jul 05 13:27:13.769657 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:13.769602 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9rcph" Jul 05 13:27:13.771167 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:13.771141 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hflvd\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="adce2775-2e72-4a05-883c-f5f80efa30ba" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd" Jul 05 13:27:13.773018 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:13.772991 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sp7mv" event={"ID":"c43e9147-0d13-403b-8e70-15aebd8de13a","Type":"ContainerStarted","Data":"faa540db109b6e70f028b9116ed9cd3402d348c0a109aed66b90e4ec602a47b1"} Jul 05 13:27:13.773111 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:13.773049 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sp7mv" event={"ID":"c43e9147-0d13-403b-8e70-15aebd8de13a","Type":"ContainerStarted","Data":"7e5f38b97759a073148911061514f7821ff92f4af11ff83c6476e7348fb1e231"} Jul 05 13:27:13.773111 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:13.773093 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sp7mv" Jul 05 13:27:13.804250 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:13.804199 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t4bhg" podStartSLOduration=1.804184626 podStartE2EDuration="1.804184626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:27:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:27:13.802076756 +0000 UTC m=+520.143094528" watchObservedRunningTime="2026-07-05 13:27:13.804184626 +0000 UTC m=+520.145202397" Jul 05 13:27:13.804719 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:13.804693 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2292t\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="f47195cc-93db-4a69-89c3-1350e929c377" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" Jul 05 13:27:13.810538 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:13.810512 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hflvd\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="adce2775-2e72-4a05-883c-f5f80efa30ba" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd" Jul 05 13:27:13.812922 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:13.812894 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2292t\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="f47195cc-93db-4a69-89c3-1350e929c377" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" Jul 05 13:27:13.845876 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:13.845830 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sp7mv" podStartSLOduration=1.845811462 podStartE2EDuration="1.845811462s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:27:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:27:13.844343916 +0000 UTC m=+520.185361685" watchObservedRunningTime="2026-07-05 13:27:13.845811462 +0000 UTC m=+520.186829230" Jul 05 13:27:13.871698 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:13.871644 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9rcph" podStartSLOduration=1.871625021 podStartE2EDuration="1.871625021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:27:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:27:13.870284661 +0000 UTC m=+520.211302433" watchObservedRunningTime="2026-07-05 13:27:13.871625021 +0000 UTC m=+520.212642792" Jul 05 13:27:14.215653 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:14.215602 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2292t\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="f47195cc-93db-4a69-89c3-1350e929c377" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2292t" Jul 05 13:27:14.217641 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:14.217615 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adce2775-2e72-4a05-883c-f5f80efa30ba" path="/var/lib/kubelet/pods/adce2775-2e72-4a05-883c-f5f80efa30ba/volumes" Jul 05 13:27:14.218022 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:14.217987 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hflvd\" is forbidden: User \"system:node:ip-10-0-141-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-119.ec2.internal' and this object" podUID="adce2775-2e72-4a05-883c-f5f80efa30ba" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflvd" Jul 05 13:27:14.218100 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:14.218088 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f47195cc-93db-4a69-89c3-1350e929c377" path="/var/lib/kubelet/pods/f47195cc-93db-4a69-89c3-1350e929c377/volumes" Jul 05 13:27:18.739803 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:18.739769 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zpsjv" Jul 05 13:27:24.781117 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:24.781040 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t4bhg" Jul 05 13:27:24.781482 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:24.781119 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9rcph" Jul 05 13:27:24.781482 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:24.781275 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sp7mv" Jul 05 13:27:24.894425 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:24.894390 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sp7mv"] Jul 05 13:27:24.894646 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:24.894620 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sp7mv" podUID="c43e9147-0d13-403b-8e70-15aebd8de13a" containerName="manager" containerID="cri-o://faa540db109b6e70f028b9116ed9cd3402d348c0a109aed66b90e4ec602a47b1" gracePeriod=10 Jul 05 13:27:25.149015 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:25.148987 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sp7mv" Jul 05 13:27:25.222473 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:25.222436 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l67zq\" (UniqueName: \"kubernetes.io/projected/c43e9147-0d13-403b-8e70-15aebd8de13a-kube-api-access-l67zq\") pod \"c43e9147-0d13-403b-8e70-15aebd8de13a\" (UID: \"c43e9147-0d13-403b-8e70-15aebd8de13a\") " Jul 05 13:27:25.222670 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:25.222480 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c43e9147-0d13-403b-8e70-15aebd8de13a-extensions-socket-volume\") pod \"c43e9147-0d13-403b-8e70-15aebd8de13a\" (UID: \"c43e9147-0d13-403b-8e70-15aebd8de13a\") " Jul 05 13:27:25.222921 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:25.222895 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c43e9147-0d13-403b-8e70-15aebd8de13a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c43e9147-0d13-403b-8e70-15aebd8de13a" (UID: "c43e9147-0d13-403b-8e70-15aebd8de13a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:27:25.224680 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:25.224657 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c43e9147-0d13-403b-8e70-15aebd8de13a-kube-api-access-l67zq" (OuterVolumeSpecName: "kube-api-access-l67zq") pod "c43e9147-0d13-403b-8e70-15aebd8de13a" (UID: "c43e9147-0d13-403b-8e70-15aebd8de13a"). InnerVolumeSpecName "kube-api-access-l67zq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:27:25.323686 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:25.323584 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l67zq\" (UniqueName: \"kubernetes.io/projected/c43e9147-0d13-403b-8e70-15aebd8de13a-kube-api-access-l67zq\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:27:25.323686 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:25.323622 2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c43e9147-0d13-403b-8e70-15aebd8de13a-extensions-socket-volume\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:27:25.843051 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:25.843014 2567 generic.go:356] "Generic (PLEG): container finished" podID="c43e9147-0d13-403b-8e70-15aebd8de13a" containerID="faa540db109b6e70f028b9116ed9cd3402d348c0a109aed66b90e4ec602a47b1" exitCode=0 Jul 05 13:27:25.843548 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:25.843089 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sp7mv" Jul 05 13:27:25.843548 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:25.843114 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sp7mv" event={"ID":"c43e9147-0d13-403b-8e70-15aebd8de13a","Type":"ContainerDied","Data":"faa540db109b6e70f028b9116ed9cd3402d348c0a109aed66b90e4ec602a47b1"} Jul 05 13:27:25.843548 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:25.843198 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sp7mv" event={"ID":"c43e9147-0d13-403b-8e70-15aebd8de13a","Type":"ContainerDied","Data":"7e5f38b97759a073148911061514f7821ff92f4af11ff83c6476e7348fb1e231"} Jul 05 13:27:25.843548 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:25.843222 2567 scope.go:117] "RemoveContainer" containerID="faa540db109b6e70f028b9116ed9cd3402d348c0a109aed66b90e4ec602a47b1" Jul 05 13:27:25.853469 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:25.853452 2567 scope.go:117] "RemoveContainer" containerID="faa540db109b6e70f028b9116ed9cd3402d348c0a109aed66b90e4ec602a47b1" Jul 05 13:27:25.853814 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:27:25.853793 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"faa540db109b6e70f028b9116ed9cd3402d348c0a109aed66b90e4ec602a47b1\": container with ID starting with faa540db109b6e70f028b9116ed9cd3402d348c0a109aed66b90e4ec602a47b1 not found: ID does not exist" containerID="faa540db109b6e70f028b9116ed9cd3402d348c0a109aed66b90e4ec602a47b1" Jul 05 13:27:25.853888 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:25.853823 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faa540db109b6e70f028b9116ed9cd3402d348c0a109aed66b90e4ec602a47b1"} err="failed to get container status \"faa540db109b6e70f028b9116ed9cd3402d348c0a109aed66b90e4ec602a47b1\": rpc error: code = NotFound desc = could not find container \"faa540db109b6e70f028b9116ed9cd3402d348c0a109aed66b90e4ec602a47b1\": container with ID starting with faa540db109b6e70f028b9116ed9cd3402d348c0a109aed66b90e4ec602a47b1 not found: ID does not exist" Jul 05 13:27:25.888920 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:25.888886 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sp7mv"] Jul 05 13:27:25.892932 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:25.892907 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sp7mv"] Jul 05 13:27:26.215428 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:26.215396 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c43e9147-0d13-403b-8e70-15aebd8de13a" path="/var/lib/kubelet/pods/c43e9147-0d13-403b-8e70-15aebd8de13a/volumes" Jul 05 13:27:41.415729 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.415694 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc"] Jul 05 13:27:41.416206 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.416191 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c43e9147-0d13-403b-8e70-15aebd8de13a" containerName="manager" Jul 05 13:27:41.416255 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.416208 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c43e9147-0d13-403b-8e70-15aebd8de13a" containerName="manager" Jul 05 13:27:41.416296 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.416275 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c43e9147-0d13-403b-8e70-15aebd8de13a" containerName="manager" Jul 05 13:27:41.422410 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.422388 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.425113 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.425095 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-mh9tq\"" Jul 05 13:27:41.433931 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.433905 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc"] Jul 05 13:27:41.471020 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.470363 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbr46\" (UniqueName: \"kubernetes.io/projected/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-kube-api-access-kbr46\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.471020 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.470426 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.471020 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.470458 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.471020 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.470493 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.471020 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.470550 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.471020 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.470616 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.471020 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.470677 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.471020 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.470728 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.471020 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.470787 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.571792 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.571726 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.571970 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.571808 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.571970 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.571845 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbr46\" (UniqueName: \"kubernetes.io/projected/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-kube-api-access-kbr46\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.571970 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.571871 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.571970 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.571901 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.571970 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.571925 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.572203 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.571976 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.572203 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.572041 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.572203 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.572108 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.572382 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.572358 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.572545 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.572514 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.572685 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.572555 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.572685 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.572525 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.572685 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.572553 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.574320 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.574290 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.574574 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.574556 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.580603 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.580578 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbr46\" (UniqueName: \"kubernetes.io/projected/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-kube-api-access-kbr46\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.580844 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.580823 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b9cdd17-b41d-4c8f-977f-7b70a30ddb94-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-hpstc\" (UID: \"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.733348 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.733255 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:41.888021 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.887981 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc"] Jul 05 13:27:41.890189 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:27:41.890161 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b9cdd17_b41d_4c8f_977f_7b70a30ddb94.slice/crio-25cf552fbf0b63b70e9c0ee50aa333370112796c01c4421cd5cc801f58b0a45b WatchSource:0}: Error finding container 25cf552fbf0b63b70e9c0ee50aa333370112796c01c4421cd5cc801f58b0a45b: Status 404 returned error can't find the container with id 25cf552fbf0b63b70e9c0ee50aa333370112796c01c4421cd5cc801f58b0a45b Jul 05 13:27:41.892940 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.892908 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:27:41.893040 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.892994 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:27:41.893107 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.893037 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:27:41.932811 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:41.932775 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" event={"ID":"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94","Type":"ContainerStarted","Data":"25cf552fbf0b63b70e9c0ee50aa333370112796c01c4421cd5cc801f58b0a45b"} Jul 05 13:27:42.940175 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:42.940141 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" event={"ID":"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94","Type":"ContainerStarted","Data":"48ca7172a8172b460d4f5680989ba4408b1fa30011c8fa052b183ccf59fbf62e"} Jul 05 13:27:42.967974 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:42.967920 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" podStartSLOduration=1.967901707 podStartE2EDuration="1.967901707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:27:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:27:42.966803388 +0000 UTC m=+549.307821161" watchObservedRunningTime="2026-07-05 13:27:42.967901707 +0000 UTC m=+549.308919478" Jul 05 13:27:43.733472 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:43.733441 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:43.738545 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:43.738510 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:43.946392 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:43.946359 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:27:43.947299 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:27:43.947280 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:28:24.265621 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:24.265586 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6bc76f999c-d9gm5"] Jul 05 13:28:24.268529 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:24.268506 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bc76f999c-d9gm5" Jul 05 13:28:24.271172 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:24.271152 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 05 13:28:24.271303 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:24.271181 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mcq5z\"" Jul 05 13:28:24.272395 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:24.272381 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 05 13:28:24.284044 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:24.284019 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bc76f999c-d9gm5"] Jul 05 13:28:24.291469 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:24.290587 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ebad9e5d-31ec-49f4-b2c0-31f62232ee1c-webhook-cert\") pod \"maas-controller-6bc76f999c-d9gm5\" (UID: \"ebad9e5d-31ec-49f4-b2c0-31f62232ee1c\") " pod="opendatahub/maas-controller-6bc76f999c-d9gm5" Jul 05 13:28:24.291469 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:24.290648 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54nsl\" (UniqueName: \"kubernetes.io/projected/ebad9e5d-31ec-49f4-b2c0-31f62232ee1c-kube-api-access-54nsl\") pod \"maas-controller-6bc76f999c-d9gm5\" (UID: \"ebad9e5d-31ec-49f4-b2c0-31f62232ee1c\") " pod="opendatahub/maas-controller-6bc76f999c-d9gm5" Jul 05 13:28:24.391511 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:24.391463 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ebad9e5d-31ec-49f4-b2c0-31f62232ee1c-webhook-cert\") pod \"maas-controller-6bc76f999c-d9gm5\" (UID: \"ebad9e5d-31ec-49f4-b2c0-31f62232ee1c\") " pod="opendatahub/maas-controller-6bc76f999c-d9gm5" Jul 05 13:28:24.391701 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:24.391520 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54nsl\" (UniqueName: \"kubernetes.io/projected/ebad9e5d-31ec-49f4-b2c0-31f62232ee1c-kube-api-access-54nsl\") pod \"maas-controller-6bc76f999c-d9gm5\" (UID: \"ebad9e5d-31ec-49f4-b2c0-31f62232ee1c\") " pod="opendatahub/maas-controller-6bc76f999c-d9gm5" Jul 05 13:28:24.394077 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:24.394046 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ebad9e5d-31ec-49f4-b2c0-31f62232ee1c-webhook-cert\") pod \"maas-controller-6bc76f999c-d9gm5\" (UID: \"ebad9e5d-31ec-49f4-b2c0-31f62232ee1c\") " pod="opendatahub/maas-controller-6bc76f999c-d9gm5" Jul 05 13:28:24.400972 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:24.400941 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54nsl\" (UniqueName: \"kubernetes.io/projected/ebad9e5d-31ec-49f4-b2c0-31f62232ee1c-kube-api-access-54nsl\") pod \"maas-controller-6bc76f999c-d9gm5\" (UID: \"ebad9e5d-31ec-49f4-b2c0-31f62232ee1c\") " pod="opendatahub/maas-controller-6bc76f999c-d9gm5" Jul 05 13:28:24.580465 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:24.580364 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bc76f999c-d9gm5" Jul 05 13:28:24.749695 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:24.749668 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bc76f999c-d9gm5"] Jul 05 13:28:24.751446 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:28:24.751420 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebad9e5d_31ec_49f4_b2c0_31f62232ee1c.slice/crio-fa487adc7c9a5a770b63e748cd02476bc83eff30996e716a86680c6c57f23a55 WatchSource:0}: Error finding container fa487adc7c9a5a770b63e748cd02476bc83eff30996e716a86680c6c57f23a55: Status 404 returned error can't find the container with id fa487adc7c9a5a770b63e748cd02476bc83eff30996e716a86680c6c57f23a55 Jul 05 13:28:25.161318 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:25.161278 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bc76f999c-d9gm5" event={"ID":"ebad9e5d-31ec-49f4-b2c0-31f62232ee1c","Type":"ContainerStarted","Data":"fa487adc7c9a5a770b63e748cd02476bc83eff30996e716a86680c6c57f23a55"} Jul 05 13:28:27.175015 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:27.174978 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bc76f999c-d9gm5" event={"ID":"ebad9e5d-31ec-49f4-b2c0-31f62232ee1c","Type":"ContainerStarted","Data":"593c1feb019a6f7051fc5827079beed1f6dfda558f2de516c26be83a0c1672c6"} Jul 05 13:28:27.175419 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:27.175104 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6bc76f999c-d9gm5" Jul 05 13:28:27.197725 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:27.197663 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6bc76f999c-d9gm5" podStartSLOduration=0.900748503 podStartE2EDuration="3.197644477s" totalImagesPullingTime="2.296895974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:28:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:28:27.195910096 +0000 UTC m=+593.536927868" watchObservedRunningTime="2026-07-05 13:28:27.197644477 +0000 UTC m=+593.538662240" Jul 05 13:28:34.127618 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:34.127585 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-26jcf_bccc5e19-26b8-4f6b-96f2-92684dc3d25a/console-operator/1.log" Jul 05 13:28:34.131800 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:34.131778 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-26jcf_bccc5e19-26b8-4f6b-96f2-92684dc3d25a/console-operator/1.log" Jul 05 13:28:34.134758 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:34.134716 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swckn_a447ce81-b3cb-4269-8d46-e6e612b89e0b/ovn-acl-logging/0.log" Jul 05 13:28:34.139098 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:34.139066 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swckn_a447ce81-b3cb-4269-8d46-e6e612b89e0b/ovn-acl-logging/0.log" Jul 05 13:28:38.186996 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:38.186959 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6bc76f999c-d9gm5" Jul 05 13:28:58.135066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:58.134978 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-lzjlh"] Jul 05 13:28:58.138894 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:58.138867 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-lzjlh" Jul 05 13:28:58.141629 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:58.141605 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-x8qz6\"" Jul 05 13:28:58.141931 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:58.141914 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 05 13:28:58.148262 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:58.148234 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-lzjlh"] Jul 05 13:28:58.167659 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:58.167625 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-4zs7b"] Jul 05 13:28:58.171317 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:58.171298 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-4zs7b" Jul 05 13:28:58.182064 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:58.182028 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-4zs7b"] Jul 05 13:28:58.335505 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:58.335468 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd314b66-0a56-4004-977c-55bd446b6ea7-plugins-config-volume\") pod \"payload-pre-processing-7964457696-lzjlh\" (UID: \"bd314b66-0a56-4004-977c-55bd446b6ea7\") " pod="openshift-ingress/payload-pre-processing-7964457696-lzjlh" Jul 05 13:28:58.335706 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:58.335538 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlctd\" (UniqueName: \"kubernetes.io/projected/bd314b66-0a56-4004-977c-55bd446b6ea7-kube-api-access-mlctd\") pod \"payload-pre-processing-7964457696-lzjlh\" (UID: \"bd314b66-0a56-4004-977c-55bd446b6ea7\") " pod="openshift-ingress/payload-pre-processing-7964457696-lzjlh" Jul 05 13:28:58.335706 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:58.335610 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft97q\" (UniqueName: \"kubernetes.io/projected/73e01b28-e127-4f30-9cc8-f1cab122f548-kube-api-access-ft97q\") pod \"payload-processing-58f86c7798-4zs7b\" (UID: \"73e01b28-e127-4f30-9cc8-f1cab122f548\") " pod="openshift-ingress/payload-processing-58f86c7798-4zs7b" Jul 05 13:28:58.335706 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:58.335651 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73e01b28-e127-4f30-9cc8-f1cab122f548-plugins-config-volume\") pod \"payload-processing-58f86c7798-4zs7b\" (UID: \"73e01b28-e127-4f30-9cc8-f1cab122f548\") " pod="openshift-ingress/payload-processing-58f86c7798-4zs7b" Jul 05 13:28:58.437312 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:58.437264 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft97q\" (UniqueName: \"kubernetes.io/projected/73e01b28-e127-4f30-9cc8-f1cab122f548-kube-api-access-ft97q\") pod \"payload-processing-58f86c7798-4zs7b\" (UID: \"73e01b28-e127-4f30-9cc8-f1cab122f548\") " pod="openshift-ingress/payload-processing-58f86c7798-4zs7b" Jul 05 13:28:58.437525 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:58.437335 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73e01b28-e127-4f30-9cc8-f1cab122f548-plugins-config-volume\") pod \"payload-processing-58f86c7798-4zs7b\" (UID: \"73e01b28-e127-4f30-9cc8-f1cab122f548\") " pod="openshift-ingress/payload-processing-58f86c7798-4zs7b" Jul 05 13:28:58.437525 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:58.437392 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd314b66-0a56-4004-977c-55bd446b6ea7-plugins-config-volume\") pod \"payload-pre-processing-7964457696-lzjlh\" (UID: \"bd314b66-0a56-4004-977c-55bd446b6ea7\") " pod="openshift-ingress/payload-pre-processing-7964457696-lzjlh" Jul 05 13:28:58.437525 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:58.437445 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlctd\" (UniqueName: \"kubernetes.io/projected/bd314b66-0a56-4004-977c-55bd446b6ea7-kube-api-access-mlctd\") pod \"payload-pre-processing-7964457696-lzjlh\" (UID: \"bd314b66-0a56-4004-977c-55bd446b6ea7\") " pod="openshift-ingress/payload-pre-processing-7964457696-lzjlh" Jul 05 13:28:58.438168 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:58.438141 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd314b66-0a56-4004-977c-55bd446b6ea7-plugins-config-volume\") pod \"payload-pre-processing-7964457696-lzjlh\" (UID: \"bd314b66-0a56-4004-977c-55bd446b6ea7\") " pod="openshift-ingress/payload-pre-processing-7964457696-lzjlh" Jul 05 13:28:58.438332 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:58.438304 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73e01b28-e127-4f30-9cc8-f1cab122f548-plugins-config-volume\") pod \"payload-processing-58f86c7798-4zs7b\" (UID: \"73e01b28-e127-4f30-9cc8-f1cab122f548\") " pod="openshift-ingress/payload-processing-58f86c7798-4zs7b" Jul 05 13:28:58.451651 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:58.451614 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlctd\" (UniqueName: \"kubernetes.io/projected/bd314b66-0a56-4004-977c-55bd446b6ea7-kube-api-access-mlctd\") pod \"payload-pre-processing-7964457696-lzjlh\" (UID: \"bd314b66-0a56-4004-977c-55bd446b6ea7\") " pod="openshift-ingress/payload-pre-processing-7964457696-lzjlh" Jul 05 13:28:58.451651 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:58.451647 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft97q\" (UniqueName: \"kubernetes.io/projected/73e01b28-e127-4f30-9cc8-f1cab122f548-kube-api-access-ft97q\") pod \"payload-processing-58f86c7798-4zs7b\" (UID: \"73e01b28-e127-4f30-9cc8-f1cab122f548\") " pod="openshift-ingress/payload-processing-58f86c7798-4zs7b" Jul 05 13:28:58.483538 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:58.483501 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-4zs7b" Jul 05 13:28:58.671756 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:58.671714 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-4zs7b"] Jul 05 13:28:58.674541 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:28:58.674512 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73e01b28_e127_4f30_9cc8_f1cab122f548.slice/crio-0fbf04e4924b057570decaba89b8307e2931d0a00e95f71f9218aaf82f0856ad WatchSource:0}: Error finding container 0fbf04e4924b057570decaba89b8307e2931d0a00e95f71f9218aaf82f0856ad: Status 404 returned error can't find the container with id 0fbf04e4924b057570decaba89b8307e2931d0a00e95f71f9218aaf82f0856ad Jul 05 13:28:58.750814 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:58.750693 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-lzjlh" Jul 05 13:28:58.947217 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:58.947164 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-lzjlh"] Jul 05 13:28:58.950308 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:28:58.950276 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd314b66_0a56_4004_977c_55bd446b6ea7.slice/crio-ead22b5cd1d240b4c98df97ec5f4f9df65c8eee0f5343f7d81a6609c933bf78d WatchSource:0}: Error finding container ead22b5cd1d240b4c98df97ec5f4f9df65c8eee0f5343f7d81a6609c933bf78d: Status 404 returned error can't find the container with id ead22b5cd1d240b4c98df97ec5f4f9df65c8eee0f5343f7d81a6609c933bf78d Jul 05 13:28:59.355336 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:59.355226 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-lzjlh" event={"ID":"bd314b66-0a56-4004-977c-55bd446b6ea7","Type":"ContainerStarted","Data":"ead22b5cd1d240b4c98df97ec5f4f9df65c8eee0f5343f7d81a6609c933bf78d"} Jul 05 13:28:59.357774 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:28:59.357105 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-4zs7b" event={"ID":"73e01b28-e127-4f30-9cc8-f1cab122f548","Type":"ContainerStarted","Data":"0fbf04e4924b057570decaba89b8307e2931d0a00e95f71f9218aaf82f0856ad"} Jul 05 13:29:00.364855 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:00.364724 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-lzjlh" event={"ID":"bd314b66-0a56-4004-977c-55bd446b6ea7","Type":"ContainerStarted","Data":"6c333347247fd495982b35f55581ebd56a5fae8041f668dbe69380526afae6fc"} Jul 05 13:29:00.364855 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:00.364812 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-lzjlh" Jul 05 13:29:00.366454 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:00.366423 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-4zs7b" event={"ID":"73e01b28-e127-4f30-9cc8-f1cab122f548","Type":"ContainerStarted","Data":"56e2c891b81ff01dbf0688b8cdc6f699304ffb791798b5b5b4387a09383ec132"} Jul 05 13:29:00.366598 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:00.366560 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-4zs7b" Jul 05 13:29:00.383765 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:00.383695 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-lzjlh" podStartSLOduration=1.252270889 podStartE2EDuration="2.383673907s" totalImagesPullingTime="1.131403018s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:28:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:29:00.382658693 +0000 UTC m=+626.723676465" watchObservedRunningTime="2026-07-05 13:29:00.383673907 +0000 UTC m=+626.724691681" Jul 05 13:29:00.400362 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:00.400282 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-4zs7b" podStartSLOduration=0.995037657 podStartE2EDuration="2.400253392s" totalImagesPullingTime="1.405215735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:28:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:29:00.398060006 +0000 UTC m=+626.739077779" watchObservedRunningTime="2026-07-05 13:29:00.400253392 +0000 UTC m=+626.741271166" Jul 05 13:29:06.400675 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:06.400638 2567 generic.go:356] "Generic (PLEG): container finished" podID="2b9cdd17-b41d-4c8f-977f-7b70a30ddb94" containerID="48ca7172a8172b460d4f5680989ba4408b1fa30011c8fa052b183ccf59fbf62e" exitCode=0 Jul 05 13:29:06.401146 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:06.400713 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" event={"ID":"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94","Type":"ContainerDied","Data":"48ca7172a8172b460d4f5680989ba4408b1fa30011c8fa052b183ccf59fbf62e"} Jul 05 13:29:06.401193 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:06.401165 2567 scope.go:117] "RemoveContainer" containerID="48ca7172a8172b460d4f5680989ba4408b1fa30011c8fa052b183ccf59fbf62e" Jul 05 13:29:06.402174 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:06.402148 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:29:06.402227 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:06.402215 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:29:06.402261 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:06.402245 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:29:07.414248 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:07.413257 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" event={"ID":"2b9cdd17-b41d-4c8f-977f-7b70a30ddb94","Type":"ContainerStarted","Data":"3e1ac02d1e085c97f9c61b72f56eed1d017591fcc0300d29d03d4e1f27c661ef"} Jul 05 13:29:07.733875 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:07.733784 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:29:07.734041 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:07.734018 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-hpstc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 05 13:29:07.734085 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:07.734069 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" podUID="2b9cdd17-b41d-4c8f-977f-7b70a30ddb94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 05 13:29:08.734604 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:08.734572 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-hpstc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 05 13:29:08.735031 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:08.734627 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" podUID="2b9cdd17-b41d-4c8f-977f-7b70a30ddb94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 05 13:29:09.734384 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:09.734351 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-hpstc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 05 13:29:09.734564 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:09.734419 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" podUID="2b9cdd17-b41d-4c8f-977f-7b70a30ddb94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 05 13:29:10.734770 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:10.734708 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-hpstc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 05 13:29:10.735167 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:10.734814 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" podUID="2b9cdd17-b41d-4c8f-977f-7b70a30ddb94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 05 13:29:11.374687 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:11.374654 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-4zs7b" Jul 05 13:29:11.374915 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:11.374844 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-lzjlh" Jul 05 13:29:11.737583 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:11.737554 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:29:11.737993 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:11.737979 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:29:11.738765 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:11.738716 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hpstc" Jul 05 13:29:37.866088 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:37.866053 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54"] Jul 05 13:29:37.869999 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:37.869979 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" Jul 05 13:29:37.873091 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:37.873063 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5hzdx\"" Jul 05 13:29:37.873230 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:37.873175 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 05 13:29:37.874484 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:37.874452 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 05 13:29:37.874644 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:37.874500 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 05 13:29:37.879198 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:37.879172 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54"] Jul 05 13:29:37.941131 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:37.941090 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/623e80be-3ca7-4872-9519-d6d70c67a29a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ddf54\" (UID: \"623e80be-3ca7-4872-9519-d6d70c67a29a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" Jul 05 13:29:37.941131 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:37.941147 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/623e80be-3ca7-4872-9519-d6d70c67a29a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ddf54\" (UID: \"623e80be-3ca7-4872-9519-d6d70c67a29a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" Jul 05 13:29:37.941357 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:37.941164 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4zwm\" (UniqueName: \"kubernetes.io/projected/623e80be-3ca7-4872-9519-d6d70c67a29a-kube-api-access-b4zwm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ddf54\" (UID: \"623e80be-3ca7-4872-9519-d6d70c67a29a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" Jul 05 13:29:37.941357 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:37.941216 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/623e80be-3ca7-4872-9519-d6d70c67a29a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ddf54\" (UID: \"623e80be-3ca7-4872-9519-d6d70c67a29a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" Jul 05 13:29:37.941357 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:37.941263 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/623e80be-3ca7-4872-9519-d6d70c67a29a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ddf54\" (UID: \"623e80be-3ca7-4872-9519-d6d70c67a29a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" Jul 05 13:29:37.941357 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:37.941307 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/623e80be-3ca7-4872-9519-d6d70c67a29a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ddf54\" (UID: \"623e80be-3ca7-4872-9519-d6d70c67a29a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" Jul 05 13:29:38.042379 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:38.042341 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/623e80be-3ca7-4872-9519-d6d70c67a29a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ddf54\" (UID: \"623e80be-3ca7-4872-9519-d6d70c67a29a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" Jul 05 13:29:38.042379 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:38.042379 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4zwm\" (UniqueName: \"kubernetes.io/projected/623e80be-3ca7-4872-9519-d6d70c67a29a-kube-api-access-b4zwm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ddf54\" (UID: \"623e80be-3ca7-4872-9519-d6d70c67a29a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" Jul 05 13:29:38.042628 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:38.042432 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/623e80be-3ca7-4872-9519-d6d70c67a29a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ddf54\" (UID: \"623e80be-3ca7-4872-9519-d6d70c67a29a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" Jul 05 13:29:38.042628 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:38.042476 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/623e80be-3ca7-4872-9519-d6d70c67a29a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ddf54\" (UID: \"623e80be-3ca7-4872-9519-d6d70c67a29a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" Jul 05 13:29:38.042628 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:38.042540 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/623e80be-3ca7-4872-9519-d6d70c67a29a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ddf54\" (UID: \"623e80be-3ca7-4872-9519-d6d70c67a29a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" Jul 05 13:29:38.042628 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:38.042578 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/623e80be-3ca7-4872-9519-d6d70c67a29a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ddf54\" (UID: \"623e80be-3ca7-4872-9519-d6d70c67a29a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" Jul 05 13:29:38.042976 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:38.042947 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/623e80be-3ca7-4872-9519-d6d70c67a29a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ddf54\" (UID: \"623e80be-3ca7-4872-9519-d6d70c67a29a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" Jul 05 13:29:38.043075 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:38.042970 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/623e80be-3ca7-4872-9519-d6d70c67a29a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ddf54\" (UID: \"623e80be-3ca7-4872-9519-d6d70c67a29a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" Jul 05 13:29:38.043136 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:38.043069 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/623e80be-3ca7-4872-9519-d6d70c67a29a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ddf54\" (UID: \"623e80be-3ca7-4872-9519-d6d70c67a29a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" Jul 05 13:29:38.045006 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:38.044983 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/623e80be-3ca7-4872-9519-d6d70c67a29a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ddf54\" (UID: \"623e80be-3ca7-4872-9519-d6d70c67a29a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" Jul 05 13:29:38.045159 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:38.045142 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/623e80be-3ca7-4872-9519-d6d70c67a29a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ddf54\" (UID: \"623e80be-3ca7-4872-9519-d6d70c67a29a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" Jul 05 13:29:38.051709 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:38.051685 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4zwm\" (UniqueName: \"kubernetes.io/projected/623e80be-3ca7-4872-9519-d6d70c67a29a-kube-api-access-b4zwm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ddf54\" (UID: \"623e80be-3ca7-4872-9519-d6d70c67a29a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" Jul 05 13:29:38.182197 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:38.182161 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" Jul 05 13:29:38.350274 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:38.350249 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54"] Jul 05 13:29:38.352382 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:29:38.352355 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod623e80be_3ca7_4872_9519_d6d70c67a29a.slice/crio-b2b53d13333670f30b93492ca1bf8a9c53f2a2cefd0406ceb80ba54a52f1fdcf WatchSource:0}: Error finding container b2b53d13333670f30b93492ca1bf8a9c53f2a2cefd0406ceb80ba54a52f1fdcf: Status 404 returned error can't find the container with id b2b53d13333670f30b93492ca1bf8a9c53f2a2cefd0406ceb80ba54a52f1fdcf Jul 05 13:29:38.594066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:38.593956 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" event={"ID":"623e80be-3ca7-4872-9519-d6d70c67a29a","Type":"ContainerStarted","Data":"b2b53d13333670f30b93492ca1bf8a9c53f2a2cefd0406ceb80ba54a52f1fdcf"} Jul 05 13:29:39.602544 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:39.602502 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" event={"ID":"623e80be-3ca7-4872-9519-d6d70c67a29a","Type":"ContainerStarted","Data":"de5e41b88302baaaf87a4cab71f3b3a168682229ac64b4c7d614cbb01542fe0c"} Jul 05 13:29:39.627163 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:39.627040 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" podStartSLOduration=1.603827441 podStartE2EDuration="2.627018017s" totalImagesPullingTime="1.023190576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:29:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:29:39.624711293 +0000 UTC m=+665.965729066" watchObservedRunningTime="2026-07-05 13:29:39.627018017 +0000 UTC m=+665.968035789" Jul 05 13:29:48.182679 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:48.182637 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" Jul 05 13:29:48.183218 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:48.182705 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" Jul 05 13:29:48.195264 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:48.195236 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" Jul 05 13:29:48.662566 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:29:48.662536 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ddf54" Jul 05 13:30:07.744873 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:07.744815 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w"] Jul 05 13:30:07.749834 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:07.749802 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" Jul 05 13:30:07.752985 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:07.752959 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 05 13:30:07.759245 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:07.759220 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w"] Jul 05 13:30:07.848253 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:07.848221 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57e047d9-14a2-40fb-8fbf-2c303ec2af37-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w\" (UID: \"57e047d9-14a2-40fb-8fbf-2c303ec2af37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" Jul 05 13:30:07.848440 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:07.848275 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57e047d9-14a2-40fb-8fbf-2c303ec2af37-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w\" (UID: \"57e047d9-14a2-40fb-8fbf-2c303ec2af37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" Jul 05 13:30:07.848440 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:07.848299 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxxkq\" (UniqueName: \"kubernetes.io/projected/57e047d9-14a2-40fb-8fbf-2c303ec2af37-kube-api-access-fxxkq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w\" (UID: \"57e047d9-14a2-40fb-8fbf-2c303ec2af37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" Jul 05 13:30:07.848515 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:07.848439 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57e047d9-14a2-40fb-8fbf-2c303ec2af37-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w\" (UID: \"57e047d9-14a2-40fb-8fbf-2c303ec2af37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" Jul 05 13:30:07.848515 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:07.848480 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57e047d9-14a2-40fb-8fbf-2c303ec2af37-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w\" (UID: \"57e047d9-14a2-40fb-8fbf-2c303ec2af37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" Jul 05 13:30:07.848586 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:07.848515 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57e047d9-14a2-40fb-8fbf-2c303ec2af37-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w\" (UID: \"57e047d9-14a2-40fb-8fbf-2c303ec2af37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" Jul 05 13:30:07.949177 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:07.949142 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57e047d9-14a2-40fb-8fbf-2c303ec2af37-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w\" (UID: \"57e047d9-14a2-40fb-8fbf-2c303ec2af37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" Jul 05 13:30:07.949393 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:07.949192 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57e047d9-14a2-40fb-8fbf-2c303ec2af37-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w\" (UID: \"57e047d9-14a2-40fb-8fbf-2c303ec2af37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" Jul 05 13:30:07.949393 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:07.949231 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57e047d9-14a2-40fb-8fbf-2c303ec2af37-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w\" (UID: \"57e047d9-14a2-40fb-8fbf-2c303ec2af37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" Jul 05 13:30:07.949393 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:07.949270 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57e047d9-14a2-40fb-8fbf-2c303ec2af37-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w\" (UID: \"57e047d9-14a2-40fb-8fbf-2c303ec2af37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" Jul 05 13:30:07.949613 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:07.949386 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57e047d9-14a2-40fb-8fbf-2c303ec2af37-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w\" (UID: \"57e047d9-14a2-40fb-8fbf-2c303ec2af37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" Jul 05 13:30:07.949613 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:07.949432 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxxkq\" (UniqueName: \"kubernetes.io/projected/57e047d9-14a2-40fb-8fbf-2c303ec2af37-kube-api-access-fxxkq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w\" (UID: \"57e047d9-14a2-40fb-8fbf-2c303ec2af37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" Jul 05 13:30:07.949700 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:07.949605 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57e047d9-14a2-40fb-8fbf-2c303ec2af37-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w\" (UID: \"57e047d9-14a2-40fb-8fbf-2c303ec2af37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" Jul 05 13:30:07.949700 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:07.949651 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57e047d9-14a2-40fb-8fbf-2c303ec2af37-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w\" (UID: \"57e047d9-14a2-40fb-8fbf-2c303ec2af37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" Jul 05 13:30:07.949838 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:07.949777 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57e047d9-14a2-40fb-8fbf-2c303ec2af37-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w\" (UID: \"57e047d9-14a2-40fb-8fbf-2c303ec2af37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" Jul 05 13:30:07.951719 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:07.951695 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57e047d9-14a2-40fb-8fbf-2c303ec2af37-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w\" (UID: \"57e047d9-14a2-40fb-8fbf-2c303ec2af37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" Jul 05 13:30:07.951987 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:07.951967 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57e047d9-14a2-40fb-8fbf-2c303ec2af37-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w\" (UID: \"57e047d9-14a2-40fb-8fbf-2c303ec2af37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" Jul 05 13:30:07.959773 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:07.959722 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxxkq\" (UniqueName: \"kubernetes.io/projected/57e047d9-14a2-40fb-8fbf-2c303ec2af37-kube-api-access-fxxkq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w\" (UID: \"57e047d9-14a2-40fb-8fbf-2c303ec2af37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" Jul 05 13:30:08.063087 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:08.062987 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" Jul 05 13:30:08.217853 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:08.217826 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w"] Jul 05 13:30:08.219719 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:30:08.219691 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57e047d9_14a2_40fb_8fbf_2c303ec2af37.slice/crio-92b580c9270ae9ae81ed83fec66dfc2e76a97c4515d64d4e475775bf9f29bdbc WatchSource:0}: Error finding container 92b580c9270ae9ae81ed83fec66dfc2e76a97c4515d64d4e475775bf9f29bdbc: Status 404 returned error can't find the container with id 92b580c9270ae9ae81ed83fec66dfc2e76a97c4515d64d4e475775bf9f29bdbc Jul 05 13:30:08.221113 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:08.221092 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 13:30:08.761911 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:08.761806 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" event={"ID":"57e047d9-14a2-40fb-8fbf-2c303ec2af37","Type":"ContainerStarted","Data":"4ad193ead9b5112c45a54e57caf87f4bc4fa75e66d7010bc57c7076528b749f0"} Jul 05 13:30:08.761911 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:08.761859 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" event={"ID":"57e047d9-14a2-40fb-8fbf-2c303ec2af37","Type":"ContainerStarted","Data":"92b580c9270ae9ae81ed83fec66dfc2e76a97c4515d64d4e475775bf9f29bdbc"} Jul 05 13:30:08.782073 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:08.782023 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" podStartSLOduration=1.592925792 podStartE2EDuration="1.782006589s" totalImagesPullingTime="189.080797ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:30:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:30:08.779716585 +0000 UTC m=+695.120734358" watchObservedRunningTime="2026-07-05 13:30:08.782006589 +0000 UTC m=+695.123024406" Jul 05 13:30:18.063399 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:18.063355 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" Jul 05 13:30:18.063933 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:18.063431 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" Jul 05 13:30:18.076418 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:18.076387 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" Jul 05 13:30:18.838428 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:30:18.838393 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w" Jul 05 13:32:45.293195 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:45.293147 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bc76f999c-d9gm5"] Jul 05 13:32:45.293863 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:45.293413 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6bc76f999c-d9gm5" podUID="ebad9e5d-31ec-49f4-b2c0-31f62232ee1c" containerName="manager" containerID="cri-o://593c1feb019a6f7051fc5827079beed1f6dfda558f2de516c26be83a0c1672c6" gracePeriod=30 Jul 05 13:32:45.563152 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:45.563122 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bc76f999c-d9gm5" Jul 05 13:32:45.657672 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:45.657633 2567 generic.go:356] "Generic (PLEG): container finished" podID="ebad9e5d-31ec-49f4-b2c0-31f62232ee1c" containerID="593c1feb019a6f7051fc5827079beed1f6dfda558f2de516c26be83a0c1672c6" exitCode=0 Jul 05 13:32:45.657869 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:45.657679 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bc76f999c-d9gm5" Jul 05 13:32:45.657869 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:45.657712 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bc76f999c-d9gm5" event={"ID":"ebad9e5d-31ec-49f4-b2c0-31f62232ee1c","Type":"ContainerDied","Data":"593c1feb019a6f7051fc5827079beed1f6dfda558f2de516c26be83a0c1672c6"} Jul 05 13:32:45.657869 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:45.657786 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bc76f999c-d9gm5" event={"ID":"ebad9e5d-31ec-49f4-b2c0-31f62232ee1c","Type":"ContainerDied","Data":"fa487adc7c9a5a770b63e748cd02476bc83eff30996e716a86680c6c57f23a55"} Jul 05 13:32:45.657869 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:45.657810 2567 scope.go:117] "RemoveContainer" containerID="593c1feb019a6f7051fc5827079beed1f6dfda558f2de516c26be83a0c1672c6" Jul 05 13:32:45.667054 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:45.667026 2567 scope.go:117] "RemoveContainer" containerID="593c1feb019a6f7051fc5827079beed1f6dfda558f2de516c26be83a0c1672c6" Jul 05 13:32:45.667372 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:32:45.667352 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"593c1feb019a6f7051fc5827079beed1f6dfda558f2de516c26be83a0c1672c6\": container with ID starting with 593c1feb019a6f7051fc5827079beed1f6dfda558f2de516c26be83a0c1672c6 not found: ID does not exist" containerID="593c1feb019a6f7051fc5827079beed1f6dfda558f2de516c26be83a0c1672c6" Jul 05 13:32:45.667418 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:45.667381 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"593c1feb019a6f7051fc5827079beed1f6dfda558f2de516c26be83a0c1672c6"} err="failed to get container status \"593c1feb019a6f7051fc5827079beed1f6dfda558f2de516c26be83a0c1672c6\": rpc error: code = NotFound desc = could not find container \"593c1feb019a6f7051fc5827079beed1f6dfda558f2de516c26be83a0c1672c6\": container with ID starting with 593c1feb019a6f7051fc5827079beed1f6dfda558f2de516c26be83a0c1672c6 not found: ID does not exist" Jul 05 13:32:45.736727 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:45.736693 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54nsl\" (UniqueName: \"kubernetes.io/projected/ebad9e5d-31ec-49f4-b2c0-31f62232ee1c-kube-api-access-54nsl\") pod \"ebad9e5d-31ec-49f4-b2c0-31f62232ee1c\" (UID: \"ebad9e5d-31ec-49f4-b2c0-31f62232ee1c\") " Jul 05 13:32:45.736930 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:45.736784 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ebad9e5d-31ec-49f4-b2c0-31f62232ee1c-webhook-cert\") pod \"ebad9e5d-31ec-49f4-b2c0-31f62232ee1c\" (UID: \"ebad9e5d-31ec-49f4-b2c0-31f62232ee1c\") " Jul 05 13:32:45.739161 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:45.739127 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebad9e5d-31ec-49f4-b2c0-31f62232ee1c-kube-api-access-54nsl" (OuterVolumeSpecName: "kube-api-access-54nsl") pod "ebad9e5d-31ec-49f4-b2c0-31f62232ee1c" (UID: "ebad9e5d-31ec-49f4-b2c0-31f62232ee1c"). InnerVolumeSpecName "kube-api-access-54nsl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:32:45.739275 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:45.739160 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebad9e5d-31ec-49f4-b2c0-31f62232ee1c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ebad9e5d-31ec-49f4-b2c0-31f62232ee1c" (UID: "ebad9e5d-31ec-49f4-b2c0-31f62232ee1c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:32:45.838609 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:45.838517 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54nsl\" (UniqueName: \"kubernetes.io/projected/ebad9e5d-31ec-49f4-b2c0-31f62232ee1c-kube-api-access-54nsl\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:32:45.838609 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:45.838548 2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ebad9e5d-31ec-49f4-b2c0-31f62232ee1c-webhook-cert\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:32:45.993458 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:45.993414 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bc76f999c-d9gm5"] Jul 05 13:32:46.005317 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:46.005287 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6bc76f999c-d9gm5"] Jul 05 13:32:46.216298 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:46.216263 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebad9e5d-31ec-49f4-b2c0-31f62232ee1c" path="/var/lib/kubelet/pods/ebad9e5d-31ec-49f4-b2c0-31f62232ee1c/volumes" Jul 05 13:32:46.828983 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:46.828942 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6bc76f999c-vr2c9"] Jul 05 13:32:46.829480 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:46.829463 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebad9e5d-31ec-49f4-b2c0-31f62232ee1c" containerName="manager" Jul 05 13:32:46.829529 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:46.829482 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebad9e5d-31ec-49f4-b2c0-31f62232ee1c" containerName="manager" Jul 05 13:32:46.829609 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:46.829599 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebad9e5d-31ec-49f4-b2c0-31f62232ee1c" containerName="manager" Jul 05 13:32:46.834269 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:46.834240 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bc76f999c-vr2c9" Jul 05 13:32:46.838523 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:46.838490 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mcq5z\"" Jul 05 13:32:46.838523 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:46.838510 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 05 13:32:46.838728 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:46.838543 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 05 13:32:46.846674 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:46.846645 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bc76f999c-vr2c9"] Jul 05 13:32:46.847209 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:46.847185 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a37fda52-258c-4b8c-8363-62b7abe1d96f-webhook-cert\") pod \"maas-controller-6bc76f999c-vr2c9\" (UID: \"a37fda52-258c-4b8c-8363-62b7abe1d96f\") " pod="opendatahub/maas-controller-6bc76f999c-vr2c9" Jul 05 13:32:46.847297 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:46.847234 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zcqr\" (UniqueName: \"kubernetes.io/projected/a37fda52-258c-4b8c-8363-62b7abe1d96f-kube-api-access-6zcqr\") pod \"maas-controller-6bc76f999c-vr2c9\" (UID: \"a37fda52-258c-4b8c-8363-62b7abe1d96f\") " pod="opendatahub/maas-controller-6bc76f999c-vr2c9" Jul 05 13:32:46.948296 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:46.948260 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a37fda52-258c-4b8c-8363-62b7abe1d96f-webhook-cert\") pod \"maas-controller-6bc76f999c-vr2c9\" (UID: \"a37fda52-258c-4b8c-8363-62b7abe1d96f\") " pod="opendatahub/maas-controller-6bc76f999c-vr2c9" Jul 05 13:32:46.948496 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:46.948314 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zcqr\" (UniqueName: \"kubernetes.io/projected/a37fda52-258c-4b8c-8363-62b7abe1d96f-kube-api-access-6zcqr\") pod \"maas-controller-6bc76f999c-vr2c9\" (UID: \"a37fda52-258c-4b8c-8363-62b7abe1d96f\") " pod="opendatahub/maas-controller-6bc76f999c-vr2c9" Jul 05 13:32:46.950919 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:46.950887 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a37fda52-258c-4b8c-8363-62b7abe1d96f-webhook-cert\") pod \"maas-controller-6bc76f999c-vr2c9\" (UID: \"a37fda52-258c-4b8c-8363-62b7abe1d96f\") " pod="opendatahub/maas-controller-6bc76f999c-vr2c9" Jul 05 13:32:46.958649 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:46.958607 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zcqr\" (UniqueName: \"kubernetes.io/projected/a37fda52-258c-4b8c-8363-62b7abe1d96f-kube-api-access-6zcqr\") pod \"maas-controller-6bc76f999c-vr2c9\" (UID: \"a37fda52-258c-4b8c-8363-62b7abe1d96f\") " pod="opendatahub/maas-controller-6bc76f999c-vr2c9" Jul 05 13:32:47.146966 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:47.146857 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bc76f999c-vr2c9" Jul 05 13:32:47.310030 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:47.310001 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bc76f999c-vr2c9"] Jul 05 13:32:47.311663 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:32:47.311630 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda37fda52_258c_4b8c_8363_62b7abe1d96f.slice/crio-f62a010de442fb198cc0f0303b1a056bfbc67880ffe726eb65d3c563d79d235b WatchSource:0}: Error finding container f62a010de442fb198cc0f0303b1a056bfbc67880ffe726eb65d3c563d79d235b: Status 404 returned error can't find the container with id f62a010de442fb198cc0f0303b1a056bfbc67880ffe726eb65d3c563d79d235b Jul 05 13:32:47.670470 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:47.670431 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bc76f999c-vr2c9" event={"ID":"a37fda52-258c-4b8c-8363-62b7abe1d96f","Type":"ContainerStarted","Data":"f62a010de442fb198cc0f0303b1a056bfbc67880ffe726eb65d3c563d79d235b"} Jul 05 13:32:48.677367 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:48.677325 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bc76f999c-vr2c9" event={"ID":"a37fda52-258c-4b8c-8363-62b7abe1d96f","Type":"ContainerStarted","Data":"7214e593916610439c2cbeed86645a66962d7a9ef85d1a0f501a465c09110cf1"} Jul 05 13:32:48.677793 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:48.677413 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6bc76f999c-vr2c9" Jul 05 13:32:48.699840 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:48.699779 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6bc76f999c-vr2c9" podStartSLOduration=2.206160627 podStartE2EDuration="2.699759791s" totalImagesPullingTime="493.599164ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:32:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:32:48.697360657 +0000 UTC m=+855.038378434" watchObservedRunningTime="2026-07-05 13:32:48.699759791 +0000 UTC m=+855.040777563" Jul 05 13:32:59.690539 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:32:59.690506 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6bc76f999c-vr2c9" Jul 05 13:33:34.165198 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:33:34.165167 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-26jcf_bccc5e19-26b8-4f6b-96f2-92684dc3d25a/console-operator/1.log" Jul 05 13:33:34.168027 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:33:34.168005 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-26jcf_bccc5e19-26b8-4f6b-96f2-92684dc3d25a/console-operator/1.log" Jul 05 13:33:34.171448 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:33:34.171427 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swckn_a447ce81-b3cb-4269-8d46-e6e612b89e0b/ovn-acl-logging/0.log" Jul 05 13:33:34.174318 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:33:34.174296 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swckn_a447ce81-b3cb-4269-8d46-e6e612b89e0b/ovn-acl-logging/0.log" Jul 05 13:37:32.603648 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:37:32.603609 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t4bhg"] Jul 05 13:37:32.604197 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:37:32.603860 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t4bhg" podUID="b2d46bf8-5c47-40bc-89e5-62d9ac618410" containerName="manager" containerID="cri-o://8356cbcc74cae9affac643c7e8a95aabf92641c483431ac7470439c2a03c7a59" gracePeriod=10 Jul 05 13:37:35.671031 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:37:35.671003 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t4bhg" Jul 05 13:37:35.790283 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:37:35.790178 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgl79\" (UniqueName: \"kubernetes.io/projected/b2d46bf8-5c47-40bc-89e5-62d9ac618410-kube-api-access-pgl79\") pod \"b2d46bf8-5c47-40bc-89e5-62d9ac618410\" (UID: \"b2d46bf8-5c47-40bc-89e5-62d9ac618410\") " Jul 05 13:37:35.790283 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:37:35.790240 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2d46bf8-5c47-40bc-89e5-62d9ac618410-extensions-socket-volume\") pod \"b2d46bf8-5c47-40bc-89e5-62d9ac618410\" (UID: \"b2d46bf8-5c47-40bc-89e5-62d9ac618410\") " Jul 05 13:37:35.790685 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:37:35.790653 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2d46bf8-5c47-40bc-89e5-62d9ac618410-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b2d46bf8-5c47-40bc-89e5-62d9ac618410" (UID: "b2d46bf8-5c47-40bc-89e5-62d9ac618410"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:37:35.792472 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:37:35.792436 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2d46bf8-5c47-40bc-89e5-62d9ac618410-kube-api-access-pgl79" (OuterVolumeSpecName: "kube-api-access-pgl79") pod "b2d46bf8-5c47-40bc-89e5-62d9ac618410" (UID: "b2d46bf8-5c47-40bc-89e5-62d9ac618410"). InnerVolumeSpecName "kube-api-access-pgl79". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:37:35.891372 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:37:35.891330 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgl79\" (UniqueName: \"kubernetes.io/projected/b2d46bf8-5c47-40bc-89e5-62d9ac618410-kube-api-access-pgl79\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:37:35.891372 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:37:35.891370 2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2d46bf8-5c47-40bc-89e5-62d9ac618410-extensions-socket-volume\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:37:36.332204 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:37:36.332170 2567 generic.go:356] "Generic (PLEG): container finished" podID="b2d46bf8-5c47-40bc-89e5-62d9ac618410" containerID="8356cbcc74cae9affac643c7e8a95aabf92641c483431ac7470439c2a03c7a59" exitCode=0 Jul 05 13:37:36.332204 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:37:36.332216 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t4bhg" Jul 05 13:37:36.332439 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:37:36.332220 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t4bhg" event={"ID":"b2d46bf8-5c47-40bc-89e5-62d9ac618410","Type":"ContainerDied","Data":"8356cbcc74cae9affac643c7e8a95aabf92641c483431ac7470439c2a03c7a59"} Jul 05 13:37:36.332439 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:37:36.332330 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t4bhg" event={"ID":"b2d46bf8-5c47-40bc-89e5-62d9ac618410","Type":"ContainerDied","Data":"f9af8dfe5d65a3d7aa629adab1c1a20bde20d55742b17b58a9d306a3288f9546"} Jul 05 13:37:36.332439 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:37:36.332352 2567 scope.go:117] "RemoveContainer" containerID="8356cbcc74cae9affac643c7e8a95aabf92641c483431ac7470439c2a03c7a59" Jul 05 13:37:36.342826 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:37:36.342766 2567 scope.go:117] "RemoveContainer" containerID="8356cbcc74cae9affac643c7e8a95aabf92641c483431ac7470439c2a03c7a59" Jul 05 13:37:36.343091 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:37:36.343073 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8356cbcc74cae9affac643c7e8a95aabf92641c483431ac7470439c2a03c7a59\": container with ID starting with 8356cbcc74cae9affac643c7e8a95aabf92641c483431ac7470439c2a03c7a59 not found: ID does not exist" containerID="8356cbcc74cae9affac643c7e8a95aabf92641c483431ac7470439c2a03c7a59" Jul 05 13:37:36.343140 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:37:36.343101 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8356cbcc74cae9affac643c7e8a95aabf92641c483431ac7470439c2a03c7a59"} err="failed to get container status \"8356cbcc74cae9affac643c7e8a95aabf92641c483431ac7470439c2a03c7a59\": rpc error: code = NotFound desc = could not find container \"8356cbcc74cae9affac643c7e8a95aabf92641c483431ac7470439c2a03c7a59\": container with ID starting with 8356cbcc74cae9affac643c7e8a95aabf92641c483431ac7470439c2a03c7a59 not found: ID does not exist" Jul 05 13:37:36.352057 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:37:36.352024 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t4bhg"] Jul 05 13:37:36.355593 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:37:36.355563 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t4bhg"] Jul 05 13:37:38.216707 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:37:38.216667 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2d46bf8-5c47-40bc-89e5-62d9ac618410" path="/var/lib/kubelet/pods/b2d46bf8-5c47-40bc-89e5-62d9ac618410/volumes" Jul 05 13:38:33.850605 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:38:33.850568 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptddt"] Jul 05 13:38:33.851115 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:38:33.851038 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2d46bf8-5c47-40bc-89e5-62d9ac618410" containerName="manager" Jul 05 13:38:33.851115 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:38:33.851051 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2d46bf8-5c47-40bc-89e5-62d9ac618410" containerName="manager" Jul 05 13:38:33.851189 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:38:33.851125 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2d46bf8-5c47-40bc-89e5-62d9ac618410" containerName="manager" Jul 05 13:38:33.854372 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:38:33.854353 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptddt" Jul 05 13:38:33.857719 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:38:33.857700 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-5tssm\"" Jul 05 13:38:33.874291 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:38:33.874259 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptddt"] Jul 05 13:38:33.972089 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:38:33.972047 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8e6f3d44-83ad-49a3-812f-dc6751f022d7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ptddt\" (UID: \"8e6f3d44-83ad-49a3-812f-dc6751f022d7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptddt" Jul 05 13:38:33.972306 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:38:33.972126 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gdw5\" (UniqueName: \"kubernetes.io/projected/8e6f3d44-83ad-49a3-812f-dc6751f022d7-kube-api-access-2gdw5\") pod \"kuadrant-operator-controller-manager-86878c769f-ptddt\" (UID: \"8e6f3d44-83ad-49a3-812f-dc6751f022d7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptddt" Jul 05 13:38:34.072807 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:38:34.072725 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8e6f3d44-83ad-49a3-812f-dc6751f022d7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ptddt\" (UID: \"8e6f3d44-83ad-49a3-812f-dc6751f022d7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptddt" Jul 05 13:38:34.072987 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:38:34.072849 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gdw5\" (UniqueName: \"kubernetes.io/projected/8e6f3d44-83ad-49a3-812f-dc6751f022d7-kube-api-access-2gdw5\") pod \"kuadrant-operator-controller-manager-86878c769f-ptddt\" (UID: \"8e6f3d44-83ad-49a3-812f-dc6751f022d7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptddt" Jul 05 13:38:34.073195 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:38:34.073169 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8e6f3d44-83ad-49a3-812f-dc6751f022d7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ptddt\" (UID: \"8e6f3d44-83ad-49a3-812f-dc6751f022d7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptddt" Jul 05 13:38:34.090276 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:38:34.090238 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gdw5\" (UniqueName: \"kubernetes.io/projected/8e6f3d44-83ad-49a3-812f-dc6751f022d7-kube-api-access-2gdw5\") pod \"kuadrant-operator-controller-manager-86878c769f-ptddt\" (UID: \"8e6f3d44-83ad-49a3-812f-dc6751f022d7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptddt" Jul 05 13:38:34.168347 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:38:34.168320 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-5tssm\"" Jul 05 13:38:34.175235 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:38:34.175206 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptddt" Jul 05 13:38:34.219114 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:38:34.219084 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-26jcf_bccc5e19-26b8-4f6b-96f2-92684dc3d25a/console-operator/1.log" Jul 05 13:38:34.220072 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:38:34.220049 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-26jcf_bccc5e19-26b8-4f6b-96f2-92684dc3d25a/console-operator/1.log" Jul 05 13:38:34.226767 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:38:34.226717 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swckn_a447ce81-b3cb-4269-8d46-e6e612b89e0b/ovn-acl-logging/0.log" Jul 05 13:38:34.227714 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:38:34.227694 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swckn_a447ce81-b3cb-4269-8d46-e6e612b89e0b/ovn-acl-logging/0.log" Jul 05 13:38:34.348636 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:38:34.348608 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptddt"] Jul 05 13:38:34.350716 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:38:34.350686 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e6f3d44_83ad_49a3_812f_dc6751f022d7.slice/crio-62ffe00b41c1866085c6682d94a805fbe929914c897b5cff53387ff980f3d5f3 WatchSource:0}: Error finding container 62ffe00b41c1866085c6682d94a805fbe929914c897b5cff53387ff980f3d5f3: Status 404 returned error can't find the container with id 62ffe00b41c1866085c6682d94a805fbe929914c897b5cff53387ff980f3d5f3 Jul 05 13:38:34.663806 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:38:34.663771 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptddt" event={"ID":"8e6f3d44-83ad-49a3-812f-dc6751f022d7","Type":"ContainerStarted","Data":"7a5764442a0c1d34f365b5fe8ecf19e91dd456df2bd4d2e7ae779baf82da3de8"} Jul 05 13:38:34.663999 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:38:34.663851 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptddt" event={"ID":"8e6f3d44-83ad-49a3-812f-dc6751f022d7","Type":"ContainerStarted","Data":"62ffe00b41c1866085c6682d94a805fbe929914c897b5cff53387ff980f3d5f3"} Jul 05 13:38:34.663999 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:38:34.663875 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptddt" Jul 05 13:38:34.684273 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:38:34.684161 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptddt" podStartSLOduration=1.6841450610000002 podStartE2EDuration="1.684145061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:38:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:38:34.682801558 +0000 UTC m=+1201.023819330" watchObservedRunningTime="2026-07-05 13:38:34.684145061 +0000 UTC m=+1201.025162832" Jul 05 13:38:45.671994 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:38:45.671959 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptddt" Jul 05 13:43:34.257785 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:43:34.257753 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-26jcf_bccc5e19-26b8-4f6b-96f2-92684dc3d25a/console-operator/1.log" Jul 05 13:43:34.261516 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:43:34.261493 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-26jcf_bccc5e19-26b8-4f6b-96f2-92684dc3d25a/console-operator/1.log" Jul 05 13:43:34.264646 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:43:34.264626 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swckn_a447ce81-b3cb-4269-8d46-e6e612b89e0b/ovn-acl-logging/0.log" Jul 05 13:43:34.268023 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:43:34.268003 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swckn_a447ce81-b3cb-4269-8d46-e6e612b89e0b/ovn-acl-logging/0.log" Jul 05 13:44:30.383099 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.383066 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2"] Jul 05 13:44:30.387371 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.387347 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.390563 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.390541 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-6a6fcb60-openshift-default-dockercfg-d2w96\"" Jul 05 13:44:30.421146 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.421108 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2"] Jul 05 13:44:30.501343 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.501304 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/803500c4-31fb-46d0-8d5b-55c612d99cfd-istiod-ca-cert\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.501530 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.501362 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt7s6\" (UniqueName: \"kubernetes.io/projected/803500c4-31fb-46d0-8d5b-55c612d99cfd-kube-api-access-gt7s6\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.501530 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.501388 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-workload-socket\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.501530 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.501439 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-istio-data\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.501530 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.501515 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-credential-socket\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.501734 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.501546 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-workload-certs\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.501734 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.501580 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/803500c4-31fb-46d0-8d5b-55c612d99cfd-istio-podinfo\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.501734 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.501674 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/803500c4-31fb-46d0-8d5b-55c612d99cfd-istio-token\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.501734 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.501714 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-istio-envoy\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.602919 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.602871 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-credential-socket\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.603128 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.602930 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-workload-certs\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.603128 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.602979 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/803500c4-31fb-46d0-8d5b-55c612d99cfd-istio-podinfo\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.603128 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.603034 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/803500c4-31fb-46d0-8d5b-55c612d99cfd-istio-token\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.603302 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.603168 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-istio-envoy\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.603302 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.603212 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/803500c4-31fb-46d0-8d5b-55c612d99cfd-istiod-ca-cert\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.603302 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.603272 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt7s6\" (UniqueName: \"kubernetes.io/projected/803500c4-31fb-46d0-8d5b-55c612d99cfd-kube-api-access-gt7s6\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.603302 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.603292 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-workload-socket\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.603504 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.603335 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-istio-data\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.603504 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.603384 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-workload-certs\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.603504 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.603394 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-credential-socket\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.603684 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.603662 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-istio-data\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.603772 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.603713 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-workload-socket\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.604041 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.604019 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/803500c4-31fb-46d0-8d5b-55c612d99cfd-istiod-ca-cert\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.605678 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.605650 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-istio-envoy\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.605829 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.605796 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/803500c4-31fb-46d0-8d5b-55c612d99cfd-istio-podinfo\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.613175 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.613144 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/803500c4-31fb-46d0-8d5b-55c612d99cfd-istio-token\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.614499 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.614478 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt7s6\" (UniqueName: \"kubernetes.io/projected/803500c4-31fb-46d0-8d5b-55c612d99cfd-kube-api-access-gt7s6\") pod \"e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.699123 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.699083 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:30.858841 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.858817 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2"] Jul 05 13:44:30.861544 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:44:30.861510 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod803500c4_31fb_46d0_8d5b_55c612d99cfd.slice/crio-75f673b493ff9d6f17ccaa2fe591e974fbf3201c83a03b494e9a2e8d2e4abc46 WatchSource:0}: Error finding container 75f673b493ff9d6f17ccaa2fe591e974fbf3201c83a03b494e9a2e8d2e4abc46: Status 404 returned error can't find the container with id 75f673b493ff9d6f17ccaa2fe591e974fbf3201c83a03b494e9a2e8d2e4abc46 Jul 05 13:44:30.863708 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.863674 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:44:30.863823 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.863775 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:44:30.863896 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:30.863827 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:44:31.702949 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:31.702909 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" event={"ID":"803500c4-31fb-46d0-8d5b-55c612d99cfd","Type":"ContainerStarted","Data":"1bdfa60f20cc6c08c5de262c7ee2b9e4dd2db29fbf7cf26593ae5b83c0afbc4c"} Jul 05 13:44:31.703327 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:31.702958 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" event={"ID":"803500c4-31fb-46d0-8d5b-55c612d99cfd","Type":"ContainerStarted","Data":"75f673b493ff9d6f17ccaa2fe591e974fbf3201c83a03b494e9a2e8d2e4abc46"} Jul 05 13:44:31.729542 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:31.729478 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" podStartSLOduration=1.7294388779999998 podStartE2EDuration="1.729438878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:44:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:44:31.728793845 +0000 UTC m=+1558.069811618" watchObservedRunningTime="2026-07-05 13:44:31.729438878 +0000 UTC m=+1558.070456651" Jul 05 13:44:32.699771 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:32.699697 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:32.704963 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:32.704935 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:32.708879 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:32.708851 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:32.709688 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:32.709667 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:36.750607 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:36.750567 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2"] Jul 05 13:44:36.751075 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:36.750781 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" podUID="803500c4-31fb-46d0-8d5b-55c612d99cfd" containerName="istio-proxy" containerID="cri-o://1bdfa60f20cc6c08c5de262c7ee2b9e4dd2db29fbf7cf26593ae5b83c0afbc4c" gracePeriod=30 Jul 05 13:44:42.020774 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.020678 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:42.122296 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.122262 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/803500c4-31fb-46d0-8d5b-55c612d99cfd-istio-podinfo\") pod \"803500c4-31fb-46d0-8d5b-55c612d99cfd\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " Jul 05 13:44:42.122491 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.122324 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gt7s6\" (UniqueName: \"kubernetes.io/projected/803500c4-31fb-46d0-8d5b-55c612d99cfd-kube-api-access-gt7s6\") pod \"803500c4-31fb-46d0-8d5b-55c612d99cfd\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " Jul 05 13:44:42.122491 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.122374 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/803500c4-31fb-46d0-8d5b-55c612d99cfd-istio-token\") pod \"803500c4-31fb-46d0-8d5b-55c612d99cfd\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " Jul 05 13:44:42.122491 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.122409 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-workload-socket\") pod \"803500c4-31fb-46d0-8d5b-55c612d99cfd\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " Jul 05 13:44:42.122491 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.122472 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-credential-socket\") pod \"803500c4-31fb-46d0-8d5b-55c612d99cfd\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " Jul 05 13:44:42.122711 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.122512 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-workload-certs\") pod \"803500c4-31fb-46d0-8d5b-55c612d99cfd\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " Jul 05 13:44:42.122711 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.122559 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/803500c4-31fb-46d0-8d5b-55c612d99cfd-istiod-ca-cert\") pod \"803500c4-31fb-46d0-8d5b-55c612d99cfd\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " Jul 05 13:44:42.122711 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.122586 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-istio-data\") pod \"803500c4-31fb-46d0-8d5b-55c612d99cfd\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " Jul 05 13:44:42.122711 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.122621 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-istio-envoy\") pod \"803500c4-31fb-46d0-8d5b-55c612d99cfd\" (UID: \"803500c4-31fb-46d0-8d5b-55c612d99cfd\") " Jul 05 13:44:42.122955 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.122774 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "803500c4-31fb-46d0-8d5b-55c612d99cfd" (UID: "803500c4-31fb-46d0-8d5b-55c612d99cfd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:44:42.122955 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.122907 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "803500c4-31fb-46d0-8d5b-55c612d99cfd" (UID: "803500c4-31fb-46d0-8d5b-55c612d99cfd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:44:42.123071 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.122992 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/803500c4-31fb-46d0-8d5b-55c612d99cfd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "803500c4-31fb-46d0-8d5b-55c612d99cfd" (UID: "803500c4-31fb-46d0-8d5b-55c612d99cfd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:44:42.123211 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.123136 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "803500c4-31fb-46d0-8d5b-55c612d99cfd" (UID: "803500c4-31fb-46d0-8d5b-55c612d99cfd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:44:42.123297 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.123275 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-istio-data" (OuterVolumeSpecName: "istio-data") pod "803500c4-31fb-46d0-8d5b-55c612d99cfd" (UID: "803500c4-31fb-46d0-8d5b-55c612d99cfd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:44:42.123417 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.123377 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-workload-socket\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:44:42.123417 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.123418 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-workload-certs\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:44:42.123612 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.123434 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/803500c4-31fb-46d0-8d5b-55c612d99cfd-istiod-ca-cert\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:44:42.124817 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.124794 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/803500c4-31fb-46d0-8d5b-55c612d99cfd-istio-token" (OuterVolumeSpecName: "istio-token") pod "803500c4-31fb-46d0-8d5b-55c612d99cfd" (UID: "803500c4-31fb-46d0-8d5b-55c612d99cfd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:44:42.124817 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.124807 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/803500c4-31fb-46d0-8d5b-55c612d99cfd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "803500c4-31fb-46d0-8d5b-55c612d99cfd" (UID: "803500c4-31fb-46d0-8d5b-55c612d99cfd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 05 13:44:42.124948 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.124818 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/803500c4-31fb-46d0-8d5b-55c612d99cfd-kube-api-access-gt7s6" (OuterVolumeSpecName: "kube-api-access-gt7s6") pod "803500c4-31fb-46d0-8d5b-55c612d99cfd" (UID: "803500c4-31fb-46d0-8d5b-55c612d99cfd"). InnerVolumeSpecName "kube-api-access-gt7s6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:44:42.124992 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.124975 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "803500c4-31fb-46d0-8d5b-55c612d99cfd" (UID: "803500c4-31fb-46d0-8d5b-55c612d99cfd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:44:42.224245 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.224220 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-istio-envoy\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:44:42.224245 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.224247 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/803500c4-31fb-46d0-8d5b-55c612d99cfd-istio-podinfo\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:44:42.224405 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.224257 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gt7s6\" (UniqueName: \"kubernetes.io/projected/803500c4-31fb-46d0-8d5b-55c612d99cfd-kube-api-access-gt7s6\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:44:42.224405 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.224266 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/803500c4-31fb-46d0-8d5b-55c612d99cfd-istio-token\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:44:42.224405 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.224276 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-credential-socket\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:44:42.224405 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.224284 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/803500c4-31fb-46d0-8d5b-55c612d99cfd-istio-data\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:44:42.770080 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.770047 2567 generic.go:356] "Generic (PLEG): container finished" podID="803500c4-31fb-46d0-8d5b-55c612d99cfd" containerID="1bdfa60f20cc6c08c5de262c7ee2b9e4dd2db29fbf7cf26593ae5b83c0afbc4c" exitCode=0 Jul 05 13:44:42.770265 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.770094 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" Jul 05 13:44:42.770265 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.770132 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" event={"ID":"803500c4-31fb-46d0-8d5b-55c612d99cfd","Type":"ContainerDied","Data":"1bdfa60f20cc6c08c5de262c7ee2b9e4dd2db29fbf7cf26593ae5b83c0afbc4c"} Jul 05 13:44:42.770265 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.770191 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2" event={"ID":"803500c4-31fb-46d0-8d5b-55c612d99cfd","Type":"ContainerDied","Data":"75f673b493ff9d6f17ccaa2fe591e974fbf3201c83a03b494e9a2e8d2e4abc46"} Jul 05 13:44:42.770265 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.770224 2567 scope.go:117] "RemoveContainer" containerID="1bdfa60f20cc6c08c5de262c7ee2b9e4dd2db29fbf7cf26593ae5b83c0afbc4c" Jul 05 13:44:42.779773 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.779735 2567 scope.go:117] "RemoveContainer" containerID="1bdfa60f20cc6c08c5de262c7ee2b9e4dd2db29fbf7cf26593ae5b83c0afbc4c" Jul 05 13:44:42.780064 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:44:42.780044 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bdfa60f20cc6c08c5de262c7ee2b9e4dd2db29fbf7cf26593ae5b83c0afbc4c\": container with ID starting with 1bdfa60f20cc6c08c5de262c7ee2b9e4dd2db29fbf7cf26593ae5b83c0afbc4c not found: ID does not exist" containerID="1bdfa60f20cc6c08c5de262c7ee2b9e4dd2db29fbf7cf26593ae5b83c0afbc4c" Jul 05 13:44:42.780143 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.780074 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bdfa60f20cc6c08c5de262c7ee2b9e4dd2db29fbf7cf26593ae5b83c0afbc4c"} err="failed to get container status \"1bdfa60f20cc6c08c5de262c7ee2b9e4dd2db29fbf7cf26593ae5b83c0afbc4c\": rpc error: code = NotFound desc = could not find container \"1bdfa60f20cc6c08c5de262c7ee2b9e4dd2db29fbf7cf26593ae5b83c0afbc4c\": container with ID starting with 1bdfa60f20cc6c08c5de262c7ee2b9e4dd2db29fbf7cf26593ae5b83c0afbc4c not found: ID does not exist" Jul 05 13:44:42.789785 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.789729 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2"] Jul 05 13:44:42.794998 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:42.794963 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-6a6fcb60-openshift-default-8688dffd4c-6v8m2"] Jul 05 13:44:44.056825 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.056791 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c"] Jul 05 13:44:44.057303 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.057289 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="803500c4-31fb-46d0-8d5b-55c612d99cfd" containerName="istio-proxy" Jul 05 13:44:44.057349 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.057306 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="803500c4-31fb-46d0-8d5b-55c612d99cfd" containerName="istio-proxy" Jul 05 13:44:44.057395 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.057386 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="803500c4-31fb-46d0-8d5b-55c612d99cfd" containerName="istio-proxy" Jul 05 13:44:44.062098 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.062075 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.065567 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.065544 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a46d09f1-openshift-default-dockercfg-qxnqd\"" Jul 05 13:44:44.080124 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.080094 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c"] Jul 05 13:44:44.216538 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.216494 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="803500c4-31fb-46d0-8d5b-55c612d99cfd" path="/var/lib/kubelet/pods/803500c4-31fb-46d0-8d5b-55c612d99cfd/volumes" Jul 05 13:44:44.246464 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.246426 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-istio-envoy\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.246654 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.246474 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe469bfb-8559-4700-974a-8256a8a33f94-istiod-ca-cert\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.246654 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.246498 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-istio-data\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.246654 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.246591 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-workload-socket\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.246654 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.246616 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-workload-certs\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.246846 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.246664 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe469bfb-8559-4700-974a-8256a8a33f94-istio-podinfo\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.246846 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.246816 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjrdp\" (UniqueName: \"kubernetes.io/projected/fe469bfb-8559-4700-974a-8256a8a33f94-kube-api-access-mjrdp\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.246914 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.246860 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-credential-socket\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.246951 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.246923 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe469bfb-8559-4700-974a-8256a8a33f94-istio-token\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.348449 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.348328 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe469bfb-8559-4700-974a-8256a8a33f94-istio-token\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.348449 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.348405 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-istio-envoy\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.348449 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.348433 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe469bfb-8559-4700-974a-8256a8a33f94-istiod-ca-cert\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.348449 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.348449 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-istio-data\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.348880 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.348497 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-workload-socket\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.348880 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.348548 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-workload-certs\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.348880 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.348645 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe469bfb-8559-4700-974a-8256a8a33f94-istio-podinfo\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.348880 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.348725 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjrdp\" (UniqueName: \"kubernetes.io/projected/fe469bfb-8559-4700-974a-8256a8a33f94-kube-api-access-mjrdp\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.348880 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.348839 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-credential-socket\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.349143 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.348987 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-istio-data\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.349143 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.349020 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-workload-socket\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.349251 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.349193 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-credential-socket\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.349251 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.349241 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-workload-certs\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.349354 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.349336 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe469bfb-8559-4700-974a-8256a8a33f94-istiod-ca-cert\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.351045 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.351023 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-istio-envoy\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.351533 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.351516 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe469bfb-8559-4700-974a-8256a8a33f94-istio-podinfo\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.357911 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.357886 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe469bfb-8559-4700-974a-8256a8a33f94-istio-token\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.358110 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.358089 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjrdp\" (UniqueName: \"kubernetes.io/projected/fe469bfb-8559-4700-974a-8256a8a33f94-kube-api-access-mjrdp\") pod \"e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.374176 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.374135 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:44.536049 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.536020 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c"] Jul 05 13:44:44.537665 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:44:44.537641 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe469bfb_8559_4700_974a_8256a8a33f94.slice/crio-3c875a339c219ff9ac005a305af20f15ad1cf3e45d4c60d77edcf730ff237abe WatchSource:0}: Error finding container 3c875a339c219ff9ac005a305af20f15ad1cf3e45d4c60d77edcf730ff237abe: Status 404 returned error can't find the container with id 3c875a339c219ff9ac005a305af20f15ad1cf3e45d4c60d77edcf730ff237abe Jul 05 13:44:44.539977 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.539944 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:44:44.540089 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.540014 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:44:44.540089 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.540044 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:44:44.786582 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.786539 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" event={"ID":"fe469bfb-8559-4700-974a-8256a8a33f94","Type":"ContainerStarted","Data":"31ddbb311b4bac20bd30669f0011aa84a1e0f02ac3f9be2dab0b151c4efc0c46"} Jul 05 13:44:44.786805 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.786611 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" event={"ID":"fe469bfb-8559-4700-974a-8256a8a33f94","Type":"ContainerStarted","Data":"3c875a339c219ff9ac005a305af20f15ad1cf3e45d4c60d77edcf730ff237abe"} Jul 05 13:44:44.809266 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:44.809194 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" podStartSLOduration=0.809169972 podStartE2EDuration="809.169972ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:44:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:44:44.807428453 +0000 UTC m=+1571.148446226" watchObservedRunningTime="2026-07-05 13:44:44.809169972 +0000 UTC m=+1571.150187744" Jul 05 13:44:45.375224 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:45.375187 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:45.380252 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:45.380219 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:45.793415 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:45.793384 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:45.794554 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:45.794527 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:56.444172 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.444133 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c"] Jul 05 13:44:56.444617 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.444355 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" podUID="fe469bfb-8559-4700-974a-8256a8a33f94" containerName="istio-proxy" containerID="cri-o://31ddbb311b4bac20bd30669f0011aa84a1e0f02ac3f9be2dab0b151c4efc0c46" gracePeriod=30 Jul 05 13:44:56.604343 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.604307 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:56.684222 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.684188 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-istio-data\") pod \"fe469bfb-8559-4700-974a-8256a8a33f94\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " Jul 05 13:44:56.684414 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.684250 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-istio-envoy\") pod \"fe469bfb-8559-4700-974a-8256a8a33f94\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " Jul 05 13:44:56.684414 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.684273 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-credential-socket\") pod \"fe469bfb-8559-4700-974a-8256a8a33f94\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " Jul 05 13:44:56.684414 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.684334 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe469bfb-8559-4700-974a-8256a8a33f94-istiod-ca-cert\") pod \"fe469bfb-8559-4700-974a-8256a8a33f94\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " Jul 05 13:44:56.684414 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.684356 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjrdp\" (UniqueName: \"kubernetes.io/projected/fe469bfb-8559-4700-974a-8256a8a33f94-kube-api-access-mjrdp\") pod \"fe469bfb-8559-4700-974a-8256a8a33f94\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " Jul 05 13:44:56.684414 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.684392 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-workload-socket\") pod \"fe469bfb-8559-4700-974a-8256a8a33f94\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " Jul 05 13:44:56.684690 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.684446 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe469bfb-8559-4700-974a-8256a8a33f94-istio-token\") pod \"fe469bfb-8559-4700-974a-8256a8a33f94\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " Jul 05 13:44:56.684690 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.684483 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe469bfb-8559-4700-974a-8256a8a33f94-istio-podinfo\") pod \"fe469bfb-8559-4700-974a-8256a8a33f94\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " Jul 05 13:44:56.684690 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.684501 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-workload-certs\") pod \"fe469bfb-8559-4700-974a-8256a8a33f94\" (UID: \"fe469bfb-8559-4700-974a-8256a8a33f94\") " Jul 05 13:44:56.685048 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.685010 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-istio-data" (OuterVolumeSpecName: "istio-data") pod "fe469bfb-8559-4700-974a-8256a8a33f94" (UID: "fe469bfb-8559-4700-974a-8256a8a33f94"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:44:56.685261 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.685223 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fe469bfb-8559-4700-974a-8256a8a33f94" (UID: "fe469bfb-8559-4700-974a-8256a8a33f94"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:44:56.685492 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.685453 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fe469bfb-8559-4700-974a-8256a8a33f94" (UID: "fe469bfb-8559-4700-974a-8256a8a33f94"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:44:56.685589 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.685571 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe469bfb-8559-4700-974a-8256a8a33f94-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fe469bfb-8559-4700-974a-8256a8a33f94" (UID: "fe469bfb-8559-4700-974a-8256a8a33f94"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:44:56.685712 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.685563 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fe469bfb-8559-4700-974a-8256a8a33f94" (UID: "fe469bfb-8559-4700-974a-8256a8a33f94"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:44:56.687065 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.687014 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe469bfb-8559-4700-974a-8256a8a33f94-kube-api-access-mjrdp" (OuterVolumeSpecName: "kube-api-access-mjrdp") pod "fe469bfb-8559-4700-974a-8256a8a33f94" (UID: "fe469bfb-8559-4700-974a-8256a8a33f94"). InnerVolumeSpecName "kube-api-access-mjrdp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:44:56.687179 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.687162 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fe469bfb-8559-4700-974a-8256a8a33f94" (UID: "fe469bfb-8559-4700-974a-8256a8a33f94"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:44:56.688079 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.688054 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe469bfb-8559-4700-974a-8256a8a33f94-istio-token" (OuterVolumeSpecName: "istio-token") pod "fe469bfb-8559-4700-974a-8256a8a33f94" (UID: "fe469bfb-8559-4700-974a-8256a8a33f94"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:44:56.688079 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.688054 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fe469bfb-8559-4700-974a-8256a8a33f94-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fe469bfb-8559-4700-974a-8256a8a33f94" (UID: "fe469bfb-8559-4700-974a-8256a8a33f94"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 05 13:44:56.785436 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.785340 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe469bfb-8559-4700-974a-8256a8a33f94-istiod-ca-cert\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:44:56.785436 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.785377 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjrdp\" (UniqueName: \"kubernetes.io/projected/fe469bfb-8559-4700-974a-8256a8a33f94-kube-api-access-mjrdp\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:44:56.785436 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.785388 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-workload-socket\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:44:56.785436 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.785397 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe469bfb-8559-4700-974a-8256a8a33f94-istio-token\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:44:56.785436 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.785407 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe469bfb-8559-4700-974a-8256a8a33f94-istio-podinfo\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:44:56.785436 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.785414 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-workload-certs\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:44:56.785436 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.785422 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-istio-data\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:44:56.785436 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.785429 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-istio-envoy\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:44:56.785436 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.785436 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe469bfb-8559-4700-974a-8256a8a33f94-credential-socket\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:44:56.868252 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.868214 2567 generic.go:356] "Generic (PLEG): container finished" podID="fe469bfb-8559-4700-974a-8256a8a33f94" containerID="31ddbb311b4bac20bd30669f0011aa84a1e0f02ac3f9be2dab0b151c4efc0c46" exitCode=0 Jul 05 13:44:56.868459 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.868266 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" Jul 05 13:44:56.868459 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.868303 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" event={"ID":"fe469bfb-8559-4700-974a-8256a8a33f94","Type":"ContainerDied","Data":"31ddbb311b4bac20bd30669f0011aa84a1e0f02ac3f9be2dab0b151c4efc0c46"} Jul 05 13:44:56.868459 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.868383 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c" event={"ID":"fe469bfb-8559-4700-974a-8256a8a33f94","Type":"ContainerDied","Data":"3c875a339c219ff9ac005a305af20f15ad1cf3e45d4c60d77edcf730ff237abe"} Jul 05 13:44:56.868459 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.868413 2567 scope.go:117] "RemoveContainer" containerID="31ddbb311b4bac20bd30669f0011aa84a1e0f02ac3f9be2dab0b151c4efc0c46" Jul 05 13:44:56.879457 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.879440 2567 scope.go:117] "RemoveContainer" containerID="31ddbb311b4bac20bd30669f0011aa84a1e0f02ac3f9be2dab0b151c4efc0c46" Jul 05 13:44:56.879814 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:44:56.879791 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31ddbb311b4bac20bd30669f0011aa84a1e0f02ac3f9be2dab0b151c4efc0c46\": container with ID starting with 31ddbb311b4bac20bd30669f0011aa84a1e0f02ac3f9be2dab0b151c4efc0c46 not found: ID does not exist" containerID="31ddbb311b4bac20bd30669f0011aa84a1e0f02ac3f9be2dab0b151c4efc0c46" Jul 05 13:44:56.879890 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.879827 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31ddbb311b4bac20bd30669f0011aa84a1e0f02ac3f9be2dab0b151c4efc0c46"} err="failed to get container status \"31ddbb311b4bac20bd30669f0011aa84a1e0f02ac3f9be2dab0b151c4efc0c46\": rpc error: code = NotFound desc = could not find container \"31ddbb311b4bac20bd30669f0011aa84a1e0f02ac3f9be2dab0b151c4efc0c46\": container with ID starting with 31ddbb311b4bac20bd30669f0011aa84a1e0f02ac3f9be2dab0b151c4efc0c46 not found: ID does not exist" Jul 05 13:44:56.893967 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.893887 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c"] Jul 05 13:44:56.896681 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:56.896650 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a46d09f1-openshift-default-6c9cd67cb5-8f54c"] Jul 05 13:44:58.217140 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:44:58.217108 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe469bfb-8559-4700-974a-8256a8a33f94" path="/var/lib/kubelet/pods/fe469bfb-8559-4700-974a-8256a8a33f94/volumes" Jul 05 13:45:03.029189 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.029155 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6"] Jul 05 13:45:03.029886 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.029618 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe469bfb-8559-4700-974a-8256a8a33f94" containerName="istio-proxy" Jul 05 13:45:03.029886 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.029630 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe469bfb-8559-4700-974a-8256a8a33f94" containerName="istio-proxy" Jul 05 13:45:03.029886 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.029729 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe469bfb-8559-4700-974a-8256a8a33f94" containerName="istio-proxy" Jul 05 13:45:03.034720 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.034697 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.037645 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.037610 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-3984164a-openshift-default-dockercfg-tnxsq\"" Jul 05 13:45:03.049769 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.049707 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6"] Jul 05 13:45:03.152148 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.152109 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istio-token\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.152353 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.152160 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-workload-certs\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.152353 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.152187 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istio-podinfo\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.152353 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.152223 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c6rt\" (UniqueName: \"kubernetes.io/projected/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-kube-api-access-5c6rt\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.152353 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.152248 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istiod-ca-cert\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.152353 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.152270 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-workload-socket\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.152537 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.152409 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istio-data\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.152537 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.152438 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istio-envoy\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.152537 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.152459 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-credential-socket\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.253621 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.253575 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istio-data\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.253621 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.253620 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istio-envoy\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.253937 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.253644 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-credential-socket\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.253937 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.253668 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istio-token\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.253937 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.253689 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-workload-certs\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.253937 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.253710 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istio-podinfo\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.253937 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.253730 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c6rt\" (UniqueName: \"kubernetes.io/projected/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-kube-api-access-5c6rt\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.253937 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.253798 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istiod-ca-cert\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.253937 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.253833 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-workload-socket\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.254292 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.254145 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istio-data\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.254352 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.254302 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-credential-socket\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.254352 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.254328 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-workload-certs\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.254510 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.254492 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-workload-socket\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.254900 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.254878 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istiod-ca-cert\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.256058 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.256038 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istio-envoy\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.256408 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.256390 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istio-podinfo\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.264182 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.264151 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istio-token\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.264455 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.264432 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c6rt\" (UniqueName: \"kubernetes.io/projected/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-kube-api-access-5c6rt\") pod \"e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.348571 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.348466 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:03.511806 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.511778 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6"] Jul 05 13:45:03.514464 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:45:03.514416 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c4cce10_1b5b_47b9_9d7b_c5f7e860fb69.slice/crio-547d550341cec055cdb5481304af78e4482f4e063934053e69928c15586bcda3 WatchSource:0}: Error finding container 547d550341cec055cdb5481304af78e4482f4e063934053e69928c15586bcda3: Status 404 returned error can't find the container with id 547d550341cec055cdb5481304af78e4482f4e063934053e69928c15586bcda3 Jul 05 13:45:03.516539 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.516504 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:45:03.516622 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.516575 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:45:03.516622 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.516604 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:45:03.913811 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.913777 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" event={"ID":"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69","Type":"ContainerStarted","Data":"9f19404fd35a11beca373f0a1e96278c6d01b51cea506fb22eb7e644a92c187c"} Jul 05 13:45:03.914006 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.913836 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" event={"ID":"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69","Type":"ContainerStarted","Data":"547d550341cec055cdb5481304af78e4482f4e063934053e69928c15586bcda3"} Jul 05 13:45:03.939288 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:03.939234 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" podStartSLOduration=0.939213003 podStartE2EDuration="939.213003ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:45:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:45:03.938231106 +0000 UTC m=+1590.279248878" watchObservedRunningTime="2026-07-05 13:45:03.939213003 +0000 UTC m=+1590.280230774" Jul 05 13:45:04.349781 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:04.349644 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:04.359541 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:04.359511 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:04.919588 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:04.919554 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:04.920515 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:04.920488 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:08.765756 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:08.765712 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6"] Jul 05 13:45:08.766169 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:08.765945 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" podUID="9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69" containerName="istio-proxy" containerID="cri-o://9f19404fd35a11beca373f0a1e96278c6d01b51cea506fb22eb7e644a92c187c" gracePeriod=30 Jul 05 13:45:15.140040 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.139961 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:15.282918 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.282880 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5c6rt\" (UniqueName: \"kubernetes.io/projected/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-kube-api-access-5c6rt\") pod \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " Jul 05 13:45:15.283107 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.282960 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-credential-socket\") pod \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " Jul 05 13:45:15.283107 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.283015 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istio-data\") pod \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " Jul 05 13:45:15.283107 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.283092 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-workload-socket\") pod \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " Jul 05 13:45:15.283274 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.283115 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istio-envoy\") pod \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " Jul 05 13:45:15.283274 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.283144 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istio-token\") pod \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " Jul 05 13:45:15.283274 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.283178 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istiod-ca-cert\") pod \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " Jul 05 13:45:15.283274 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.283225 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-workload-certs\") pod \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " Jul 05 13:45:15.283274 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.283247 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istio-podinfo\") pod \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\" (UID: \"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69\") " Jul 05 13:45:15.283528 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.283316 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69" (UID: "9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:45:15.283528 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.283390 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69" (UID: "9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:45:15.283646 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.283587 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69" (UID: "9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:45:15.283733 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.283702 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69" (UID: "9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:45:15.283840 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.283820 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-credential-socket\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:45:15.283899 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.283846 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-workload-socket\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:45:15.283899 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.283863 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-workload-certs\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:45:15.284258 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.284237 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istio-data" (OuterVolumeSpecName: "istio-data") pod "9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69" (UID: "9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:45:15.285492 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.285470 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69" (UID: "9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:45:15.285597 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.285579 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69" (UID: "9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 05 13:45:15.285869 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.285844 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istio-token" (OuterVolumeSpecName: "istio-token") pod "9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69" (UID: "9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:45:15.285925 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.285890 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-kube-api-access-5c6rt" (OuterVolumeSpecName: "kube-api-access-5c6rt") pod "9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69" (UID: "9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69"). InnerVolumeSpecName "kube-api-access-5c6rt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:45:15.385426 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.385383 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istio-podinfo\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:45:15.385426 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.385416 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5c6rt\" (UniqueName: \"kubernetes.io/projected/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-kube-api-access-5c6rt\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:45:15.385426 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.385427 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istio-data\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:45:15.385426 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.385437 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istio-envoy\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:45:15.385700 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.385444 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istio-token\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:45:15.385700 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.385452 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69-istiod-ca-cert\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:45:15.989322 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.989280 2567 generic.go:356] "Generic (PLEG): container finished" podID="9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69" containerID="9f19404fd35a11beca373f0a1e96278c6d01b51cea506fb22eb7e644a92c187c" exitCode=0 Jul 05 13:45:15.990191 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.989360 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" event={"ID":"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69","Type":"ContainerDied","Data":"9f19404fd35a11beca373f0a1e96278c6d01b51cea506fb22eb7e644a92c187c"} Jul 05 13:45:15.990363 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.989376 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" Jul 05 13:45:15.990363 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.990285 2567 scope.go:117] "RemoveContainer" containerID="9f19404fd35a11beca373f0a1e96278c6d01b51cea506fb22eb7e644a92c187c" Jul 05 13:45:15.990496 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:15.990427 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6" event={"ID":"9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69","Type":"ContainerDied","Data":"547d550341cec055cdb5481304af78e4482f4e063934053e69928c15586bcda3"} Jul 05 13:45:16.001032 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:16.001011 2567 scope.go:117] "RemoveContainer" containerID="9f19404fd35a11beca373f0a1e96278c6d01b51cea506fb22eb7e644a92c187c" Jul 05 13:45:16.001353 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:45:16.001330 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f19404fd35a11beca373f0a1e96278c6d01b51cea506fb22eb7e644a92c187c\": container with ID starting with 9f19404fd35a11beca373f0a1e96278c6d01b51cea506fb22eb7e644a92c187c not found: ID does not exist" containerID="9f19404fd35a11beca373f0a1e96278c6d01b51cea506fb22eb7e644a92c187c" Jul 05 13:45:16.001428 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:16.001366 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f19404fd35a11beca373f0a1e96278c6d01b51cea506fb22eb7e644a92c187c"} err="failed to get container status \"9f19404fd35a11beca373f0a1e96278c6d01b51cea506fb22eb7e644a92c187c\": rpc error: code = NotFound desc = could not find container \"9f19404fd35a11beca373f0a1e96278c6d01b51cea506fb22eb7e644a92c187c\": container with ID starting with 9f19404fd35a11beca373f0a1e96278c6d01b51cea506fb22eb7e644a92c187c not found: ID does not exist" Jul 05 13:45:16.027159 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:16.027121 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6"] Jul 05 13:45:16.035025 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:16.034992 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-3984164a-openshift-default-79c45784cf-kjtq6"] Jul 05 13:45:16.217828 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:45:16.217791 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69" path="/var/lib/kubelet/pods/9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69/volumes" Jul 05 13:46:54.805998 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.805923 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t"] Jul 05 13:46:54.806472 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.806451 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69" containerName="istio-proxy" Jul 05 13:46:54.806575 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.806474 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69" containerName="istio-proxy" Jul 05 13:46:54.806651 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.806634 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c4cce10-1b5b-47b9-9d7b-c5f7e860fb69" containerName="istio-proxy" Jul 05 13:46:54.809867 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.809849 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:54.815883 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.815860 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-cab5d0f5-openshift-default-dockercfg-9c42n\"" Jul 05 13:46:54.847538 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.847501 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t"] Jul 05 13:46:54.895679 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.895638 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5prkn\" (UniqueName: \"kubernetes.io/projected/0e29c19c-2fc9-4471-b8ad-738f50184512-kube-api-access-5prkn\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:54.895679 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.895693 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e29c19c-2fc9-4471-b8ad-738f50184512-istio-token\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:54.895978 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.895783 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e29c19c-2fc9-4471-b8ad-738f50184512-istio-podinfo\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:54.895978 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.895807 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e29c19c-2fc9-4471-b8ad-738f50184512-istiod-ca-cert\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:54.895978 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.895826 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-workload-socket\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:54.895978 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.895850 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-workload-certs\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:54.895978 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.895871 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-istio-data\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:54.895978 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.895913 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-credential-socket\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:54.895978 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.895943 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-istio-envoy\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:54.997086 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.997046 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e29c19c-2fc9-4471-b8ad-738f50184512-istio-token\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:54.997267 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.997139 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e29c19c-2fc9-4471-b8ad-738f50184512-istio-podinfo\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:54.997267 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.997179 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e29c19c-2fc9-4471-b8ad-738f50184512-istiod-ca-cert\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:54.997267 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.997207 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-workload-socket\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:54.997267 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.997246 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-workload-certs\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:54.997478 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.997270 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-istio-data\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:54.997478 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.997328 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-credential-socket\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:54.997478 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.997373 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-istio-envoy\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:54.997478 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.997446 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5prkn\" (UniqueName: \"kubernetes.io/projected/0e29c19c-2fc9-4471-b8ad-738f50184512-kube-api-access-5prkn\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:54.997845 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.997809 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-workload-certs\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:54.997979 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.997871 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-istio-data\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:54.998028 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.997982 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-credential-socket\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:54.998106 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.998083 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-workload-socket\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:54.998216 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.998196 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e29c19c-2fc9-4471-b8ad-738f50184512-istiod-ca-cert\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:54.999862 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.999841 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-istio-envoy\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:54.999966 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:54.999872 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e29c19c-2fc9-4471-b8ad-738f50184512-istio-podinfo\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:55.007964 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:55.007934 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5prkn\" (UniqueName: \"kubernetes.io/projected/0e29c19c-2fc9-4471-b8ad-738f50184512-kube-api-access-5prkn\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:55.010136 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:55.010111 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e29c19c-2fc9-4471-b8ad-738f50184512-istio-token\") pod \"e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:55.121129 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:55.121031 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:55.316296 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:55.316267 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t"] Jul 05 13:46:55.316813 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:46:55.316784 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e29c19c_2fc9_4471_b8ad_738f50184512.slice/crio-21564726a438ca8c9695969ba7004922aac12b01dfa591370b7358111c196c5e WatchSource:0}: Error finding container 21564726a438ca8c9695969ba7004922aac12b01dfa591370b7358111c196c5e: Status 404 returned error can't find the container with id 21564726a438ca8c9695969ba7004922aac12b01dfa591370b7358111c196c5e Jul 05 13:46:55.319008 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:55.318975 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:46:55.319093 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:55.319040 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:46:55.319093 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:55.319071 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:46:55.560249 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:55.560205 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" event={"ID":"0e29c19c-2fc9-4471-b8ad-738f50184512","Type":"ContainerStarted","Data":"9e9a2b5572d90fdbac7ee15abfb9305a99c7b3faa6687916235b7349e9c6e27a"} Jul 05 13:46:55.560443 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:55.560275 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" event={"ID":"0e29c19c-2fc9-4471-b8ad-738f50184512","Type":"ContainerStarted","Data":"21564726a438ca8c9695969ba7004922aac12b01dfa591370b7358111c196c5e"} Jul 05 13:46:55.584038 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:55.583971 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" podStartSLOduration=1.583946187 podStartE2EDuration="1.583946187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:46:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:46:55.581339206 +0000 UTC m=+1701.922357007" watchObservedRunningTime="2026-07-05 13:46:55.583946187 +0000 UTC m=+1701.924963960" Jul 05 13:46:56.121529 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:56.121483 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:56.126800 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:56.126770 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:56.565735 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:56.565702 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:46:56.566647 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:46:56.566629 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:47:11.567237 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:11.567196 2567 patch_prober.go:28] interesting pod/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 05 13:47:11.567733 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:11.567268 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" podUID="0e29c19c-2fc9-4471-b8ad-738f50184512" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 05 13:47:11.666928 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:11.666893 2567 generic.go:356] "Generic (PLEG): container finished" podID="0e29c19c-2fc9-4471-b8ad-738f50184512" containerID="9e9a2b5572d90fdbac7ee15abfb9305a99c7b3faa6687916235b7349e9c6e27a" exitCode=0 Jul 05 13:47:11.667101 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:11.666974 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" event={"ID":"0e29c19c-2fc9-4471-b8ad-738f50184512","Type":"ContainerDied","Data":"9e9a2b5572d90fdbac7ee15abfb9305a99c7b3faa6687916235b7349e9c6e27a"} Jul 05 13:47:11.667419 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:11.667404 2567 scope.go:117] "RemoveContainer" containerID="9e9a2b5572d90fdbac7ee15abfb9305a99c7b3faa6687916235b7349e9c6e27a" Jul 05 13:47:11.668594 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:11.668563 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:47:11.668666 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:11.668652 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:47:11.668708 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:11.668690 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:47:12.674105 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:12.674068 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" event={"ID":"0e29c19c-2fc9-4471-b8ad-738f50184512","Type":"ContainerStarted","Data":"d39eef0b5491b68516617116193c84ba7fe7654cb82f33b513e0f61840c009ab"} Jul 05 13:47:13.122246 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:13.122146 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:47:13.122471 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:13.122446 2567 patch_prober.go:28] interesting pod/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 05 13:47:13.122522 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:13.122498 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" podUID="0e29c19c-2fc9-4471-b8ad-738f50184512" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 05 13:47:14.122166 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:14.122120 2567 patch_prober.go:28] interesting pod/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 05 13:47:14.122558 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:14.122187 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" podUID="0e29c19c-2fc9-4471-b8ad-738f50184512" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 05 13:47:15.122149 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:15.122106 2567 patch_prober.go:28] interesting pod/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 05 13:47:15.122340 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:15.122181 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" podUID="0e29c19c-2fc9-4471-b8ad-738f50184512" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 05 13:47:16.122423 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:16.122384 2567 patch_prober.go:28] interesting pod/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 05 13:47:16.122868 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:16.122475 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" podUID="0e29c19c-2fc9-4471-b8ad-738f50184512" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 05 13:47:17.065837 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:17.065794 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t"] Jul 05 13:47:17.066023 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:17.065998 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" podUID="0e29c19c-2fc9-4471-b8ad-738f50184512" containerName="istio-proxy" containerID="cri-o://d39eef0b5491b68516617116193c84ba7fe7654cb82f33b513e0f61840c009ab" gracePeriod=30 Jul 05 13:47:22.347316 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.347240 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:47:22.490500 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.490466 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5prkn\" (UniqueName: \"kubernetes.io/projected/0e29c19c-2fc9-4471-b8ad-738f50184512-kube-api-access-5prkn\") pod \"0e29c19c-2fc9-4471-b8ad-738f50184512\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " Jul 05 13:47:22.490699 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.490508 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e29c19c-2fc9-4471-b8ad-738f50184512-istio-podinfo\") pod \"0e29c19c-2fc9-4471-b8ad-738f50184512\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " Jul 05 13:47:22.490699 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.490551 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e29c19c-2fc9-4471-b8ad-738f50184512-istiod-ca-cert\") pod \"0e29c19c-2fc9-4471-b8ad-738f50184512\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " Jul 05 13:47:22.490699 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.490573 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e29c19c-2fc9-4471-b8ad-738f50184512-istio-token\") pod \"0e29c19c-2fc9-4471-b8ad-738f50184512\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " Jul 05 13:47:22.490699 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.490656 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-workload-socket\") pod \"0e29c19c-2fc9-4471-b8ad-738f50184512\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " Jul 05 13:47:22.490966 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.490712 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-workload-certs\") pod \"0e29c19c-2fc9-4471-b8ad-738f50184512\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " Jul 05 13:47:22.490966 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.490779 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-credential-socket\") pod \"0e29c19c-2fc9-4471-b8ad-738f50184512\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " Jul 05 13:47:22.490966 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.490812 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-istio-data\") pod \"0e29c19c-2fc9-4471-b8ad-738f50184512\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " Jul 05 13:47:22.490966 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.490841 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-istio-envoy\") pod \"0e29c19c-2fc9-4471-b8ad-738f50184512\" (UID: \"0e29c19c-2fc9-4471-b8ad-738f50184512\") " Jul 05 13:47:22.491165 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.491050 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0e29c19c-2fc9-4471-b8ad-738f50184512" (UID: "0e29c19c-2fc9-4471-b8ad-738f50184512"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:47:22.491165 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.491066 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0e29c19c-2fc9-4471-b8ad-738f50184512" (UID: "0e29c19c-2fc9-4471-b8ad-738f50184512"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:47:22.491165 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.491124 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0e29c19c-2fc9-4471-b8ad-738f50184512" (UID: "0e29c19c-2fc9-4471-b8ad-738f50184512"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:47:22.491165 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.491131 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e29c19c-2fc9-4471-b8ad-738f50184512-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0e29c19c-2fc9-4471-b8ad-738f50184512" (UID: "0e29c19c-2fc9-4471-b8ad-738f50184512"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:47:22.491388 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.491356 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e29c19c-2fc9-4471-b8ad-738f50184512-istiod-ca-cert\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:47:22.491388 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.491381 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-workload-socket\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:47:22.491484 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.491395 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-workload-certs\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:47:22.491484 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.491410 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-credential-socket\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:47:22.491795 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.491772 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-istio-data" (OuterVolumeSpecName: "istio-data") pod "0e29c19c-2fc9-4471-b8ad-738f50184512" (UID: "0e29c19c-2fc9-4471-b8ad-738f50184512"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:47:22.493051 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.493027 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0e29c19c-2fc9-4471-b8ad-738f50184512-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0e29c19c-2fc9-4471-b8ad-738f50184512" (UID: "0e29c19c-2fc9-4471-b8ad-738f50184512"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 05 13:47:22.493139 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.493067 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e29c19c-2fc9-4471-b8ad-738f50184512-istio-token" (OuterVolumeSpecName: "istio-token") pod "0e29c19c-2fc9-4471-b8ad-738f50184512" (UID: "0e29c19c-2fc9-4471-b8ad-738f50184512"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:47:22.493840 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.493819 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e29c19c-2fc9-4471-b8ad-738f50184512-kube-api-access-5prkn" (OuterVolumeSpecName: "kube-api-access-5prkn") pod "0e29c19c-2fc9-4471-b8ad-738f50184512" (UID: "0e29c19c-2fc9-4471-b8ad-738f50184512"). InnerVolumeSpecName "kube-api-access-5prkn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:47:22.493840 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.493826 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0e29c19c-2fc9-4471-b8ad-738f50184512" (UID: "0e29c19c-2fc9-4471-b8ad-738f50184512"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:47:22.592545 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.592498 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-istio-data\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:47:22.592545 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.592536 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e29c19c-2fc9-4471-b8ad-738f50184512-istio-envoy\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:47:22.592545 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.592548 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5prkn\" (UniqueName: \"kubernetes.io/projected/0e29c19c-2fc9-4471-b8ad-738f50184512-kube-api-access-5prkn\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:47:22.592868 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.592564 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e29c19c-2fc9-4471-b8ad-738f50184512-istio-podinfo\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:47:22.592868 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.592576 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e29c19c-2fc9-4471-b8ad-738f50184512-istio-token\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:47:22.733714 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.733682 2567 generic.go:356] "Generic (PLEG): container finished" podID="0e29c19c-2fc9-4471-b8ad-738f50184512" containerID="d39eef0b5491b68516617116193c84ba7fe7654cb82f33b513e0f61840c009ab" exitCode=0 Jul 05 13:47:22.733956 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.733724 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" Jul 05 13:47:22.733956 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.733772 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" event={"ID":"0e29c19c-2fc9-4471-b8ad-738f50184512","Type":"ContainerDied","Data":"d39eef0b5491b68516617116193c84ba7fe7654cb82f33b513e0f61840c009ab"} Jul 05 13:47:22.733956 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.733836 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t" event={"ID":"0e29c19c-2fc9-4471-b8ad-738f50184512","Type":"ContainerDied","Data":"21564726a438ca8c9695969ba7004922aac12b01dfa591370b7358111c196c5e"} Jul 05 13:47:22.733956 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.733871 2567 scope.go:117] "RemoveContainer" containerID="d39eef0b5491b68516617116193c84ba7fe7654cb82f33b513e0f61840c009ab" Jul 05 13:47:22.744653 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.744633 2567 scope.go:117] "RemoveContainer" containerID="9e9a2b5572d90fdbac7ee15abfb9305a99c7b3faa6687916235b7349e9c6e27a" Jul 05 13:47:22.754598 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.754576 2567 scope.go:117] "RemoveContainer" containerID="d39eef0b5491b68516617116193c84ba7fe7654cb82f33b513e0f61840c009ab" Jul 05 13:47:22.754953 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:47:22.754933 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d39eef0b5491b68516617116193c84ba7fe7654cb82f33b513e0f61840c009ab\": container with ID starting with d39eef0b5491b68516617116193c84ba7fe7654cb82f33b513e0f61840c009ab not found: ID does not exist" containerID="d39eef0b5491b68516617116193c84ba7fe7654cb82f33b513e0f61840c009ab" Jul 05 13:47:22.755022 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.754963 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d39eef0b5491b68516617116193c84ba7fe7654cb82f33b513e0f61840c009ab"} err="failed to get container status \"d39eef0b5491b68516617116193c84ba7fe7654cb82f33b513e0f61840c009ab\": rpc error: code = NotFound desc = could not find container \"d39eef0b5491b68516617116193c84ba7fe7654cb82f33b513e0f61840c009ab\": container with ID starting with d39eef0b5491b68516617116193c84ba7fe7654cb82f33b513e0f61840c009ab not found: ID does not exist" Jul 05 13:47:22.755022 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.754988 2567 scope.go:117] "RemoveContainer" containerID="9e9a2b5572d90fdbac7ee15abfb9305a99c7b3faa6687916235b7349e9c6e27a" Jul 05 13:47:22.755251 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:47:22.755232 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e9a2b5572d90fdbac7ee15abfb9305a99c7b3faa6687916235b7349e9c6e27a\": container with ID starting with 9e9a2b5572d90fdbac7ee15abfb9305a99c7b3faa6687916235b7349e9c6e27a not found: ID does not exist" containerID="9e9a2b5572d90fdbac7ee15abfb9305a99c7b3faa6687916235b7349e9c6e27a" Jul 05 13:47:22.755294 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.755258 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e9a2b5572d90fdbac7ee15abfb9305a99c7b3faa6687916235b7349e9c6e27a"} err="failed to get container status \"9e9a2b5572d90fdbac7ee15abfb9305a99c7b3faa6687916235b7349e9c6e27a\": rpc error: code = NotFound desc = could not find container \"9e9a2b5572d90fdbac7ee15abfb9305a99c7b3faa6687916235b7349e9c6e27a\": container with ID starting with 9e9a2b5572d90fdbac7ee15abfb9305a99c7b3faa6687916235b7349e9c6e27a not found: ID does not exist" Jul 05 13:47:22.759490 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.759458 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t"] Jul 05 13:47:22.766143 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:22.766116 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-cab5d0f5-openshift-default-6cb85dcbff-tf29t"] Jul 05 13:47:24.216085 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:24.216044 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e29c19c-2fc9-4471-b8ad-738f50184512" path="/var/lib/kubelet/pods/0e29c19c-2fc9-4471-b8ad-738f50184512/volumes" Jul 05 13:47:38.255630 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.255595 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g"] Jul 05 13:47:38.256071 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.256058 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e29c19c-2fc9-4471-b8ad-738f50184512" containerName="istio-proxy" Jul 05 13:47:38.256129 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.256073 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e29c19c-2fc9-4471-b8ad-738f50184512" containerName="istio-proxy" Jul 05 13:47:38.256129 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.256084 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e29c19c-2fc9-4471-b8ad-738f50184512" containerName="istio-proxy" Jul 05 13:47:38.256129 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.256089 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e29c19c-2fc9-4471-b8ad-738f50184512" containerName="istio-proxy" Jul 05 13:47:38.256227 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.256173 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e29c19c-2fc9-4471-b8ad-738f50184512" containerName="istio-proxy" Jul 05 13:47:38.256227 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.256183 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e29c19c-2fc9-4471-b8ad-738f50184512" containerName="istio-proxy" Jul 05 13:47:38.259087 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.259067 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.270531 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.270502 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-3536bbfe-openshift-default-dockercfg-phq78\"" Jul 05 13:47:38.284174 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.284137 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g"] Jul 05 13:47:38.455205 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.455162 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istio-podinfo\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.455425 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.455216 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-workload-socket\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.455425 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.455302 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-workload-certs\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.455425 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.455392 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p7l7\" (UniqueName: \"kubernetes.io/projected/56ab28a6-ace5-4fad-ab27-add58de9b8b0-kube-api-access-8p7l7\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.455583 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.455449 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istio-data\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.455583 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.455478 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istio-token\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.455583 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.455530 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istiod-ca-cert\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.455583 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.455551 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istio-envoy\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.455807 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.455608 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-credential-socket\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.556636 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.556540 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istiod-ca-cert\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.556636 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.556584 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istio-envoy\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.556867 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.556715 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-credential-socket\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.556867 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.556773 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istio-podinfo\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.556867 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.556804 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-workload-socket\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.557004 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.556910 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-workload-certs\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.557004 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.556990 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p7l7\" (UniqueName: \"kubernetes.io/projected/56ab28a6-ace5-4fad-ab27-add58de9b8b0-kube-api-access-8p7l7\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.557109 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.557040 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istio-data\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.557109 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.557079 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-credential-socket\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.557219 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.557085 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istio-token\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.557277 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.557243 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-workload-socket\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.557277 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.557265 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-workload-certs\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.557470 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.557445 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istiod-ca-cert\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.557683 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.557661 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istio-data\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.559192 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.559168 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istio-envoy\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.559317 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.559300 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istio-podinfo\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.570617 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.570584 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p7l7\" (UniqueName: \"kubernetes.io/projected/56ab28a6-ace5-4fad-ab27-add58de9b8b0-kube-api-access-8p7l7\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.580429 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.580398 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istio-token\") pod \"e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:38.870718 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:38.870626 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:39.033080 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:39.033047 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g"] Jul 05 13:47:39.037474 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:47:39.037444 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56ab28a6_ace5_4fad_ab27_add58de9b8b0.slice/crio-11199ff4fa85111b0beae7bd3b93afa6870f412a59865f149ac75bf58570f740 WatchSource:0}: Error finding container 11199ff4fa85111b0beae7bd3b93afa6870f412a59865f149ac75bf58570f740: Status 404 returned error can't find the container with id 11199ff4fa85111b0beae7bd3b93afa6870f412a59865f149ac75bf58570f740 Jul 05 13:47:39.039961 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:39.039932 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:47:39.040035 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:39.040011 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:47:39.040074 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:39.040049 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:47:39.832411 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:39.832368 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" event={"ID":"56ab28a6-ace5-4fad-ab27-add58de9b8b0","Type":"ContainerStarted","Data":"dd5fe8aa2cc0d3e4f0850300022099de59824b4bc3d409be0532ac493b311408"} Jul 05 13:47:39.832856 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:39.832434 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" event={"ID":"56ab28a6-ace5-4fad-ab27-add58de9b8b0","Type":"ContainerStarted","Data":"11199ff4fa85111b0beae7bd3b93afa6870f412a59865f149ac75bf58570f740"} Jul 05 13:47:39.866091 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:39.866035 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" podStartSLOduration=1.866016375 podStartE2EDuration="1.866016375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:47:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:47:39.862901962 +0000 UTC m=+1746.203919735" watchObservedRunningTime="2026-07-05 13:47:39.866016375 +0000 UTC m=+1746.207034147" Jul 05 13:47:39.871184 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:39.871154 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:40.875990 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:40.875962 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:41.844956 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:41.844925 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:41.845931 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:41.845913 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:54.933097 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:54.933062 2567 generic.go:356] "Generic (PLEG): container finished" podID="56ab28a6-ace5-4fad-ab27-add58de9b8b0" containerID="dd5fe8aa2cc0d3e4f0850300022099de59824b4bc3d409be0532ac493b311408" exitCode=0 Jul 05 13:47:54.933523 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:54.933132 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" event={"ID":"56ab28a6-ace5-4fad-ab27-add58de9b8b0","Type":"ContainerDied","Data":"dd5fe8aa2cc0d3e4f0850300022099de59824b4bc3d409be0532ac493b311408"} Jul 05 13:47:54.933621 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:54.933607 2567 scope.go:117] "RemoveContainer" containerID="dd5fe8aa2cc0d3e4f0850300022099de59824b4bc3d409be0532ac493b311408" Jul 05 13:47:54.934624 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:54.934596 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:47:54.934728 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:54.934660 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:47:54.934728 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:54.934692 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:47:55.940661 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:55.940617 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" event={"ID":"56ab28a6-ace5-4fad-ab27-add58de9b8b0","Type":"ContainerStarted","Data":"5cd9a48115c26ee15ea530b9314c93085c10ff532ed47d8fd016e6f8634e320c"} Jul 05 13:47:56.845190 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:56.845141 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:56.871210 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:56.871174 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:47:56.872357 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:56.872318 2567 patch_prober.go:28] interesting pod/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 05 13:47:56.872589 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:56.872562 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" podUID="56ab28a6-ace5-4fad-ab27-add58de9b8b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 05 13:47:57.871087 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:57.871052 2567 patch_prober.go:28] interesting pod/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 05 13:47:57.871485 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:57.871108 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" podUID="56ab28a6-ace5-4fad-ab27-add58de9b8b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 05 13:47:58.871283 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:58.871243 2567 patch_prober.go:28] interesting pod/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 05 13:47:58.871710 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:58.871304 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" podUID="56ab28a6-ace5-4fad-ab27-add58de9b8b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 05 13:47:59.871351 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:59.871309 2567 patch_prober.go:28] interesting pod/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 05 13:47:59.871794 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:47:59.871380 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" podUID="56ab28a6-ace5-4fad-ab27-add58de9b8b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 05 13:48:00.875805 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:48:00.875736 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:48:00.876869 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:48:00.876852 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:48:34.303474 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:48:34.303440 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-26jcf_bccc5e19-26b8-4f6b-96f2-92684dc3d25a/console-operator/1.log" Jul 05 13:48:34.306714 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:48:34.306687 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-26jcf_bccc5e19-26b8-4f6b-96f2-92684dc3d25a/console-operator/1.log" Jul 05 13:48:34.310728 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:48:34.310705 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swckn_a447ce81-b3cb-4269-8d46-e6e612b89e0b/ovn-acl-logging/0.log" Jul 05 13:48:34.313932 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:48:34.313912 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swckn_a447ce81-b3cb-4269-8d46-e6e612b89e0b/ovn-acl-logging/0.log" Jul 05 13:49:10.100670 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:10.100631 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g"] Jul 05 13:49:10.101295 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:10.100913 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" podUID="56ab28a6-ace5-4fad-ab27-add58de9b8b0" containerName="istio-proxy" containerID="cri-o://5cd9a48115c26ee15ea530b9314c93085c10ff532ed47d8fd016e6f8634e320c" gracePeriod=30 Jul 05 13:49:15.371705 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.371634 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:49:15.421478 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.421445 2567 generic.go:356] "Generic (PLEG): container finished" podID="56ab28a6-ace5-4fad-ab27-add58de9b8b0" containerID="5cd9a48115c26ee15ea530b9314c93085c10ff532ed47d8fd016e6f8634e320c" exitCode=0 Jul 05 13:49:15.421643 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.421557 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" Jul 05 13:49:15.421643 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.421592 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" event={"ID":"56ab28a6-ace5-4fad-ab27-add58de9b8b0","Type":"ContainerDied","Data":"5cd9a48115c26ee15ea530b9314c93085c10ff532ed47d8fd016e6f8634e320c"} Jul 05 13:49:15.421831 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.421659 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g" event={"ID":"56ab28a6-ace5-4fad-ab27-add58de9b8b0","Type":"ContainerDied","Data":"11199ff4fa85111b0beae7bd3b93afa6870f412a59865f149ac75bf58570f740"} Jul 05 13:49:15.421831 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.421693 2567 scope.go:117] "RemoveContainer" containerID="5cd9a48115c26ee15ea530b9314c93085c10ff532ed47d8fd016e6f8634e320c" Jul 05 13:49:15.431706 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.431687 2567 scope.go:117] "RemoveContainer" containerID="dd5fe8aa2cc0d3e4f0850300022099de59824b4bc3d409be0532ac493b311408" Jul 05 13:49:15.441039 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.441017 2567 scope.go:117] "RemoveContainer" containerID="5cd9a48115c26ee15ea530b9314c93085c10ff532ed47d8fd016e6f8634e320c" Jul 05 13:49:15.441360 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:49:15.441339 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cd9a48115c26ee15ea530b9314c93085c10ff532ed47d8fd016e6f8634e320c\": container with ID starting with 5cd9a48115c26ee15ea530b9314c93085c10ff532ed47d8fd016e6f8634e320c not found: ID does not exist" containerID="5cd9a48115c26ee15ea530b9314c93085c10ff532ed47d8fd016e6f8634e320c" Jul 05 13:49:15.441410 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.441372 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cd9a48115c26ee15ea530b9314c93085c10ff532ed47d8fd016e6f8634e320c"} err="failed to get container status \"5cd9a48115c26ee15ea530b9314c93085c10ff532ed47d8fd016e6f8634e320c\": rpc error: code = NotFound desc = could not find container \"5cd9a48115c26ee15ea530b9314c93085c10ff532ed47d8fd016e6f8634e320c\": container with ID starting with 5cd9a48115c26ee15ea530b9314c93085c10ff532ed47d8fd016e6f8634e320c not found: ID does not exist" Jul 05 13:49:15.441410 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.441392 2567 scope.go:117] "RemoveContainer" containerID="dd5fe8aa2cc0d3e4f0850300022099de59824b4bc3d409be0532ac493b311408" Jul 05 13:49:15.441660 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:49:15.441637 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd5fe8aa2cc0d3e4f0850300022099de59824b4bc3d409be0532ac493b311408\": container with ID starting with dd5fe8aa2cc0d3e4f0850300022099de59824b4bc3d409be0532ac493b311408 not found: ID does not exist" containerID="dd5fe8aa2cc0d3e4f0850300022099de59824b4bc3d409be0532ac493b311408" Jul 05 13:49:15.441700 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.441670 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd5fe8aa2cc0d3e4f0850300022099de59824b4bc3d409be0532ac493b311408"} err="failed to get container status \"dd5fe8aa2cc0d3e4f0850300022099de59824b4bc3d409be0532ac493b311408\": rpc error: code = NotFound desc = could not find container \"dd5fe8aa2cc0d3e4f0850300022099de59824b4bc3d409be0532ac493b311408\": container with ID starting with dd5fe8aa2cc0d3e4f0850300022099de59824b4bc3d409be0532ac493b311408 not found: ID does not exist" Jul 05 13:49:15.463832 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.463796 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istio-podinfo\") pod \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " Jul 05 13:49:15.463973 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.463875 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-workload-certs\") pod \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " Jul 05 13:49:15.463973 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.463910 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-workload-socket\") pod \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " Jul 05 13:49:15.463973 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.463934 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-credential-socket\") pod \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " Jul 05 13:49:15.463973 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.463961 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istio-token\") pod \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " Jul 05 13:49:15.464180 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.464056 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istio-data\") pod \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " Jul 05 13:49:15.464180 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.464084 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8p7l7\" (UniqueName: \"kubernetes.io/projected/56ab28a6-ace5-4fad-ab27-add58de9b8b0-kube-api-access-8p7l7\") pod \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " Jul 05 13:49:15.464180 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.464117 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istio-envoy\") pod \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " Jul 05 13:49:15.464180 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.464145 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istiod-ca-cert\") pod \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\" (UID: \"56ab28a6-ace5-4fad-ab27-add58de9b8b0\") " Jul 05 13:49:15.464393 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.464248 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "56ab28a6-ace5-4fad-ab27-add58de9b8b0" (UID: "56ab28a6-ace5-4fad-ab27-add58de9b8b0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:49:15.464393 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.464259 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "56ab28a6-ace5-4fad-ab27-add58de9b8b0" (UID: "56ab28a6-ace5-4fad-ab27-add58de9b8b0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:49:15.464393 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.464277 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "56ab28a6-ace5-4fad-ab27-add58de9b8b0" (UID: "56ab28a6-ace5-4fad-ab27-add58de9b8b0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:49:15.464632 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.464613 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-workload-certs\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:49:15.464697 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.464638 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-workload-socket\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:49:15.464697 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.464651 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-credential-socket\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:49:15.464998 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.464861 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "56ab28a6-ace5-4fad-ab27-add58de9b8b0" (UID: "56ab28a6-ace5-4fad-ab27-add58de9b8b0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:49:15.465584 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.465550 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istio-data" (OuterVolumeSpecName: "istio-data") pod "56ab28a6-ace5-4fad-ab27-add58de9b8b0" (UID: "56ab28a6-ace5-4fad-ab27-add58de9b8b0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:49:15.466272 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.466253 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "56ab28a6-ace5-4fad-ab27-add58de9b8b0" (UID: "56ab28a6-ace5-4fad-ab27-add58de9b8b0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 05 13:49:15.466365 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.466348 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "56ab28a6-ace5-4fad-ab27-add58de9b8b0" (UID: "56ab28a6-ace5-4fad-ab27-add58de9b8b0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:49:15.466537 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.466521 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istio-token" (OuterVolumeSpecName: "istio-token") pod "56ab28a6-ace5-4fad-ab27-add58de9b8b0" (UID: "56ab28a6-ace5-4fad-ab27-add58de9b8b0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:49:15.467013 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.466983 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56ab28a6-ace5-4fad-ab27-add58de9b8b0-kube-api-access-8p7l7" (OuterVolumeSpecName: "kube-api-access-8p7l7") pod "56ab28a6-ace5-4fad-ab27-add58de9b8b0" (UID: "56ab28a6-ace5-4fad-ab27-add58de9b8b0"). InnerVolumeSpecName "kube-api-access-8p7l7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:49:15.565666 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.565622 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istio-podinfo\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:49:15.565666 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.565658 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istio-token\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:49:15.565666 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.565668 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istio-data\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:49:15.565666 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.565677 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8p7l7\" (UniqueName: \"kubernetes.io/projected/56ab28a6-ace5-4fad-ab27-add58de9b8b0-kube-api-access-8p7l7\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:49:15.566058 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.565686 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istio-envoy\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:49:15.566058 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.565695 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56ab28a6-ace5-4fad-ab27-add58de9b8b0-istiod-ca-cert\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:49:15.748539 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.748505 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g"] Jul 05 13:49:15.754893 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:15.754864 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-3536bbfe-openshift-default-77f4d7cdb7-dm54g"] Jul 05 13:49:16.217120 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:49:16.217085 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56ab28a6-ace5-4fad-ab27-add58de9b8b0" path="/var/lib/kubelet/pods/56ab28a6-ace5-4fad-ab27-add58de9b8b0/volumes" Jul 05 13:50:09.507243 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.507152 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd"] Jul 05 13:50:09.507774 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.507698 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56ab28a6-ace5-4fad-ab27-add58de9b8b0" containerName="istio-proxy" Jul 05 13:50:09.507774 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.507711 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="56ab28a6-ace5-4fad-ab27-add58de9b8b0" containerName="istio-proxy" Jul 05 13:50:09.507774 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.507734 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56ab28a6-ace5-4fad-ab27-add58de9b8b0" containerName="istio-proxy" Jul 05 13:50:09.507774 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.507758 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="56ab28a6-ace5-4fad-ab27-add58de9b8b0" containerName="istio-proxy" Jul 05 13:50:09.507922 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.507825 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="56ab28a6-ace5-4fad-ab27-add58de9b8b0" containerName="istio-proxy" Jul 05 13:50:09.511215 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.511193 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.514287 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.514261 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-76289d-openshift-default-dockercfg-nc67x\"" Jul 05 13:50:09.539907 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.539870 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd"] Jul 05 13:50:09.608442 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.608401 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4grq2\" (UniqueName: \"kubernetes.io/projected/3370c679-bd5a-49c7-bd54-5967cd8c9da1-kube-api-access-4grq2\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.608634 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.608465 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-credential-socket\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.608634 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.608521 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istio-data\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.608634 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.608595 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istiod-ca-cert\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.608634 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.608626 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-workload-certs\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.608887 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.608649 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istio-envoy\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.608887 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.608674 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istio-token\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.608887 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.608732 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-workload-socket\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.608887 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.608788 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istio-podinfo\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.709449 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.709407 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istio-data\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.709659 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.709475 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istiod-ca-cert\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.709659 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.709501 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-workload-certs\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.709659 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.709526 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istio-envoy\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.709659 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.709556 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istio-token\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.709659 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.709606 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-workload-socket\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.709659 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.709645 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istio-podinfo\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.710019 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.709685 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4grq2\" (UniqueName: \"kubernetes.io/projected/3370c679-bd5a-49c7-bd54-5967cd8c9da1-kube-api-access-4grq2\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.710019 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.709772 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-credential-socket\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.710019 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.709897 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istio-data\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.710019 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.709976 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-workload-certs\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.710198 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.710180 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istiod-ca-cert\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.710300 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.710282 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-workload-socket\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.710363 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.710319 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-credential-socket\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.712265 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.712243 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istio-envoy\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.712423 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.712407 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istio-podinfo\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.722470 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.722434 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istio-token\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.734031 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.733998 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4grq2\" (UniqueName: \"kubernetes.io/projected/3370c679-bd5a-49c7-bd54-5967cd8c9da1-kube-api-access-4grq2\") pod \"e2e-inf-a-76289d-openshift-default-84976548df-67wwd\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.822592 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.822479 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:09.985731 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.985696 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd"] Jul 05 13:50:09.988882 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:50:09.988852 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3370c679_bd5a_49c7_bd54_5967cd8c9da1.slice/crio-6c986f22c446fd2eeee321178c6e714695227574bee13484f32b4358b5a4a8b8 WatchSource:0}: Error finding container 6c986f22c446fd2eeee321178c6e714695227574bee13484f32b4358b5a4a8b8: Status 404 returned error can't find the container with id 6c986f22c446fd2eeee321178c6e714695227574bee13484f32b4358b5a4a8b8 Jul 05 13:50:09.991513 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.991474 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:50:09.991630 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.991548 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:50:09.991630 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:09.991578 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:50:10.734725 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:10.734679 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" event={"ID":"3370c679-bd5a-49c7-bd54-5967cd8c9da1","Type":"ContainerStarted","Data":"bcd3265beaeed47e07cb478f21b1daeecc97249dee4bd91fcc51c833dd4f7a4c"} Jul 05 13:50:10.735215 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:10.734763 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" event={"ID":"3370c679-bd5a-49c7-bd54-5967cd8c9da1","Type":"ContainerStarted","Data":"6c986f22c446fd2eeee321178c6e714695227574bee13484f32b4358b5a4a8b8"} Jul 05 13:50:10.764498 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:10.764435 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" podStartSLOduration=1.764414554 podStartE2EDuration="1.764414554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:50:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:50:10.76164648 +0000 UTC m=+1897.102664253" watchObservedRunningTime="2026-07-05 13:50:10.764414554 +0000 UTC m=+1897.105432330" Jul 05 13:50:10.822805 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:10.822768 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:11.828600 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:11.828566 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:12.745636 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:12.745602 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:12.746732 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:12.746709 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:17.925088 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:17.924963 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv"] Jul 05 13:50:17.926167 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:17.925861 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="56ab28a6-ace5-4fad-ab27-add58de9b8b0" containerName="istio-proxy" Jul 05 13:50:17.929784 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:17.929729 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:17.932529 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:17.932501 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-4b5487-openshift-default-dockercfg-k742w\"" Jul 05 13:50:17.947654 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:17.947627 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv"] Jul 05 13:50:18.004896 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.004849 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istiod-ca-cert\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.005099 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.004909 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-workload-certs\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.005099 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.005016 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istio-envoy\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.005099 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.005076 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-workload-socket\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.005228 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.005105 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-credential-socket\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.005228 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.005125 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istio-data\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.005228 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.005152 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istio-token\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.005228 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.005181 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istio-podinfo\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.005358 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.005227 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfp6b\" (UniqueName: \"kubernetes.io/projected/c7b41d70-7920-4dc4-aea3-44bc85a4480a-kube-api-access-sfp6b\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.106644 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.106599 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istiod-ca-cert\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.107119 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.107090 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-workload-certs\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.107261 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.107182 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istio-envoy\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.107261 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.107251 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-workload-socket\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.107382 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.107287 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-credential-socket\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.107382 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.107314 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istio-data\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.107495 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.107392 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istio-token\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.107495 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.107426 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istio-podinfo\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.107495 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.107458 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfp6b\" (UniqueName: \"kubernetes.io/projected/c7b41d70-7920-4dc4-aea3-44bc85a4480a-kube-api-access-sfp6b\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.107495 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.107457 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istiod-ca-cert\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.108065 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.107780 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-workload-certs\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.108065 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.107844 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-credential-socket\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.108483 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.108440 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istio-data\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.109140 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.109111 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-workload-socket\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.110766 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.110599 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istio-envoy\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.111238 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.111213 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istio-podinfo\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.118867 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.118817 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfp6b\" (UniqueName: \"kubernetes.io/projected/c7b41d70-7920-4dc4-aea3-44bc85a4480a-kube-api-access-sfp6b\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.120583 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.120521 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istio-token\") pod \"e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.241540 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.241436 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:18.396979 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.396935 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv"] Jul 05 13:50:18.398328 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:50:18.398295 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7b41d70_7920_4dc4_aea3_44bc85a4480a.slice/crio-c65ffad8af7c22e0fb67eec3540820a4e41a1e255e6b5f7279f14cf8ca9da8cf WatchSource:0}: Error finding container c65ffad8af7c22e0fb67eec3540820a4e41a1e255e6b5f7279f14cf8ca9da8cf: Status 404 returned error can't find the container with id c65ffad8af7c22e0fb67eec3540820a4e41a1e255e6b5f7279f14cf8ca9da8cf Jul 05 13:50:18.400804 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.400766 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:50:18.400875 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.400858 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:50:18.400912 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.400900 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:50:18.784543 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.784506 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" event={"ID":"c7b41d70-7920-4dc4-aea3-44bc85a4480a","Type":"ContainerStarted","Data":"ef10f405be214ff64d158747387240f5b76b10af052e704ae03868bab4fb85ca"} Jul 05 13:50:18.784780 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.784559 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" event={"ID":"c7b41d70-7920-4dc4-aea3-44bc85a4480a","Type":"ContainerStarted","Data":"c65ffad8af7c22e0fb67eec3540820a4e41a1e255e6b5f7279f14cf8ca9da8cf"} Jul 05 13:50:18.808189 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:18.808126 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" podStartSLOduration=1.808109315 podStartE2EDuration="1.808109315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:50:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:50:18.807822998 +0000 UTC m=+1905.148840773" watchObservedRunningTime="2026-07-05 13:50:18.808109315 +0000 UTC m=+1905.149127091" Jul 05 13:50:19.242144 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:19.242103 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:19.247211 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:19.247185 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:19.793427 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:19.793394 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:19.794500 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:19.794478 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:28.458251 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:28.457023 2567 patch_prober.go:28] interesting pod/e2e-inf-a-76289d-openshift-default-84976548df-67wwd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": read tcp 10.132.0.2:53974->10.132.0.57:15021: read: connection reset by peer" start-of-body= Jul 05 13:50:28.458251 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:28.457110 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" podUID="3370c679-bd5a-49c7-bd54-5967cd8c9da1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": read tcp 10.132.0.2:53974->10.132.0.57:15021: read: connection reset by peer" Jul 05 13:50:28.862529 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:28.862438 2567 generic.go:356] "Generic (PLEG): container finished" podID="3370c679-bd5a-49c7-bd54-5967cd8c9da1" containerID="bcd3265beaeed47e07cb478f21b1daeecc97249dee4bd91fcc51c833dd4f7a4c" exitCode=0 Jul 05 13:50:28.862529 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:28.862507 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" event={"ID":"3370c679-bd5a-49c7-bd54-5967cd8c9da1","Type":"ContainerDied","Data":"bcd3265beaeed47e07cb478f21b1daeecc97249dee4bd91fcc51c833dd4f7a4c"} Jul 05 13:50:28.862974 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:28.862958 2567 scope.go:117] "RemoveContainer" containerID="bcd3265beaeed47e07cb478f21b1daeecc97249dee4bd91fcc51c833dd4f7a4c" Jul 05 13:50:28.864041 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:28.864017 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:50:28.864102 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:28.864089 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:50:28.864139 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:28.864125 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:50:29.869816 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:29.869778 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" event={"ID":"3370c679-bd5a-49c7-bd54-5967cd8c9da1","Type":"ContainerStarted","Data":"185e2ef0dd40865e0571096a6bc4651086b85b58f7b157a35d9c15b3e231d0cb"} Jul 05 13:50:30.822999 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:30.822947 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:30.823428 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:30.823395 2567 patch_prober.go:28] interesting pod/e2e-inf-a-76289d-openshift-default-84976548df-67wwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 05 13:50:30.823564 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:30.823455 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" podUID="3370c679-bd5a-49c7-bd54-5967cd8c9da1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 05 13:50:31.823646 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:31.823600 2567 patch_prober.go:28] interesting pod/e2e-inf-a-76289d-openshift-default-84976548df-67wwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 05 13:50:31.824292 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:31.824253 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" podUID="3370c679-bd5a-49c7-bd54-5967cd8c9da1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 05 13:50:32.823196 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:32.823156 2567 patch_prober.go:28] interesting pod/e2e-inf-a-76289d-openshift-default-84976548df-67wwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 05 13:50:32.823532 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:32.823484 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" podUID="3370c679-bd5a-49c7-bd54-5967cd8c9da1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 05 13:50:33.823629 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:33.823577 2567 patch_prober.go:28] interesting pod/e2e-inf-a-76289d-openshift-default-84976548df-67wwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 05 13:50:33.824181 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:33.823666 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" podUID="3370c679-bd5a-49c7-bd54-5967cd8c9da1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 05 13:50:34.843532 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:34.843500 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:34.844060 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:34.843958 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:34.844843 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:34.844817 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:50:35.834973 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:35.834931 2567 patch_prober.go:28] interesting pod/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 05 13:50:35.835167 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:35.835012 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" podUID="c7b41d70-7920-4dc4-aea3-44bc85a4480a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 05 13:50:37.925386 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:37.925347 2567 generic.go:356] "Generic (PLEG): container finished" podID="c7b41d70-7920-4dc4-aea3-44bc85a4480a" containerID="ef10f405be214ff64d158747387240f5b76b10af052e704ae03868bab4fb85ca" exitCode=0 Jul 05 13:50:37.925872 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:37.925427 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" event={"ID":"c7b41d70-7920-4dc4-aea3-44bc85a4480a","Type":"ContainerDied","Data":"ef10f405be214ff64d158747387240f5b76b10af052e704ae03868bab4fb85ca"} Jul 05 13:50:37.925969 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:37.925954 2567 scope.go:117] "RemoveContainer" containerID="ef10f405be214ff64d158747387240f5b76b10af052e704ae03868bab4fb85ca" Jul 05 13:50:37.927205 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:37.927175 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:50:37.927330 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:37.927310 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:50:37.927399 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:37.927366 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:50:38.933886 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:38.933850 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" event={"ID":"c7b41d70-7920-4dc4-aea3-44bc85a4480a","Type":"ContainerStarted","Data":"8ba918ca06a83b26925419b6e880aaec845c9fab1ffbc5645b3c5b3f3d825d16"} Jul 05 13:50:39.241887 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:39.241784 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:39.242103 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:39.242081 2567 patch_prober.go:28] interesting pod/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 05 13:50:39.242189 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:39.242139 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" podUID="c7b41d70-7920-4dc4-aea3-44bc85a4480a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 05 13:50:40.242363 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:40.242328 2567 patch_prober.go:28] interesting pod/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 05 13:50:40.242782 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:40.242388 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" podUID="c7b41d70-7920-4dc4-aea3-44bc85a4480a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 05 13:50:41.242233 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:41.242193 2567 patch_prober.go:28] interesting pod/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 05 13:50:41.242475 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:41.242260 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" podUID="c7b41d70-7920-4dc4-aea3-44bc85a4480a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 05 13:50:42.242204 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:42.242163 2567 patch_prober.go:28] interesting pod/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 05 13:50:42.242400 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:42.242226 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" podUID="c7b41d70-7920-4dc4-aea3-44bc85a4480a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 05 13:50:43.245975 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:43.245937 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:43.246372 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:43.246228 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:50:43.247073 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:50:43.247049 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:52:18.148953 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:18.148917 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd"] Jul 05 13:52:18.149555 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:18.149188 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" podUID="3370c679-bd5a-49c7-bd54-5967cd8c9da1" containerName="istio-proxy" containerID="cri-o://185e2ef0dd40865e0571096a6bc4651086b85b58f7b157a35d9c15b3e231d0cb" gracePeriod=30 Jul 05 13:52:19.846875 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:19.846836 2567 patch_prober.go:28] interesting pod/e2e-inf-a-76289d-openshift-default-84976548df-67wwd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 05 13:52:19.847251 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:19.846901 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" podUID="3370c679-bd5a-49c7-bd54-5967cd8c9da1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 05 13:52:23.420422 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.420349 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:52:23.426353 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.426325 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istiod-ca-cert\") pod \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " Jul 05 13:52:23.426521 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.426381 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istio-data\") pod \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " Jul 05 13:52:23.426521 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.426413 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-workload-socket\") pod \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " Jul 05 13:52:23.426521 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.426446 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istio-envoy\") pod \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " Jul 05 13:52:23.426521 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.426474 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-credential-socket\") pod \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " Jul 05 13:52:23.426521 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.426496 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-workload-certs\") pod \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " Jul 05 13:52:23.426521 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.426522 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istio-podinfo\") pod \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " Jul 05 13:52:23.426878 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.426585 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4grq2\" (UniqueName: \"kubernetes.io/projected/3370c679-bd5a-49c7-bd54-5967cd8c9da1-kube-api-access-4grq2\") pod \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " Jul 05 13:52:23.426878 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.426646 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istio-token\") pod \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\" (UID: \"3370c679-bd5a-49c7-bd54-5967cd8c9da1\") " Jul 05 13:52:23.426878 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.426715 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3370c679-bd5a-49c7-bd54-5967cd8c9da1" (UID: "3370c679-bd5a-49c7-bd54-5967cd8c9da1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:52:23.426878 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.426794 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3370c679-bd5a-49c7-bd54-5967cd8c9da1" (UID: "3370c679-bd5a-49c7-bd54-5967cd8c9da1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:52:23.427106 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.426877 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3370c679-bd5a-49c7-bd54-5967cd8c9da1" (UID: "3370c679-bd5a-49c7-bd54-5967cd8c9da1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:52:23.427106 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.427031 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3370c679-bd5a-49c7-bd54-5967cd8c9da1" (UID: "3370c679-bd5a-49c7-bd54-5967cd8c9da1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:52:23.427242 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.427208 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istiod-ca-cert\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:52:23.427242 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.427231 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-workload-socket\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:52:23.427345 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.427244 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-credential-socket\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:52:23.427345 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.427257 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-workload-certs\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:52:23.427510 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.427447 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istio-data" (OuterVolumeSpecName: "istio-data") pod "3370c679-bd5a-49c7-bd54-5967cd8c9da1" (UID: "3370c679-bd5a-49c7-bd54-5967cd8c9da1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:52:23.429147 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.429115 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3370c679-bd5a-49c7-bd54-5967cd8c9da1" (UID: "3370c679-bd5a-49c7-bd54-5967cd8c9da1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:52:23.429321 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.429302 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istio-token" (OuterVolumeSpecName: "istio-token") pod "3370c679-bd5a-49c7-bd54-5967cd8c9da1" (UID: "3370c679-bd5a-49c7-bd54-5967cd8c9da1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:52:23.429528 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.429509 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3370c679-bd5a-49c7-bd54-5967cd8c9da1" (UID: "3370c679-bd5a-49c7-bd54-5967cd8c9da1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 05 13:52:23.429586 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.429527 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3370c679-bd5a-49c7-bd54-5967cd8c9da1-kube-api-access-4grq2" (OuterVolumeSpecName: "kube-api-access-4grq2") pod "3370c679-bd5a-49c7-bd54-5967cd8c9da1" (UID: "3370c679-bd5a-49c7-bd54-5967cd8c9da1"). InnerVolumeSpecName "kube-api-access-4grq2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:52:23.528849 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.528806 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istio-data\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:52:23.528849 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.528843 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istio-envoy\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:52:23.528849 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.528855 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istio-podinfo\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:52:23.529111 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.528867 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4grq2\" (UniqueName: \"kubernetes.io/projected/3370c679-bd5a-49c7-bd54-5967cd8c9da1-kube-api-access-4grq2\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:52:23.529111 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.528880 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3370c679-bd5a-49c7-bd54-5967cd8c9da1-istio-token\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:52:23.545530 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.545488 2567 generic.go:356] "Generic (PLEG): container finished" podID="3370c679-bd5a-49c7-bd54-5967cd8c9da1" containerID="185e2ef0dd40865e0571096a6bc4651086b85b58f7b157a35d9c15b3e231d0cb" exitCode=0 Jul 05 13:52:23.545728 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.545542 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" Jul 05 13:52:23.545728 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.545568 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" event={"ID":"3370c679-bd5a-49c7-bd54-5967cd8c9da1","Type":"ContainerDied","Data":"185e2ef0dd40865e0571096a6bc4651086b85b58f7b157a35d9c15b3e231d0cb"} Jul 05 13:52:23.545728 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.545625 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd" event={"ID":"3370c679-bd5a-49c7-bd54-5967cd8c9da1","Type":"ContainerDied","Data":"6c986f22c446fd2eeee321178c6e714695227574bee13484f32b4358b5a4a8b8"} Jul 05 13:52:23.545728 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.545650 2567 scope.go:117] "RemoveContainer" containerID="185e2ef0dd40865e0571096a6bc4651086b85b58f7b157a35d9c15b3e231d0cb" Jul 05 13:52:23.556329 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.556288 2567 scope.go:117] "RemoveContainer" containerID="bcd3265beaeed47e07cb478f21b1daeecc97249dee4bd91fcc51c833dd4f7a4c" Jul 05 13:52:23.566275 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.566252 2567 scope.go:117] "RemoveContainer" containerID="185e2ef0dd40865e0571096a6bc4651086b85b58f7b157a35d9c15b3e231d0cb" Jul 05 13:52:23.566618 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:52:23.566593 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"185e2ef0dd40865e0571096a6bc4651086b85b58f7b157a35d9c15b3e231d0cb\": container with ID starting with 185e2ef0dd40865e0571096a6bc4651086b85b58f7b157a35d9c15b3e231d0cb not found: ID does not exist" containerID="185e2ef0dd40865e0571096a6bc4651086b85b58f7b157a35d9c15b3e231d0cb" Jul 05 13:52:23.566678 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.566628 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"185e2ef0dd40865e0571096a6bc4651086b85b58f7b157a35d9c15b3e231d0cb"} err="failed to get container status \"185e2ef0dd40865e0571096a6bc4651086b85b58f7b157a35d9c15b3e231d0cb\": rpc error: code = NotFound desc = could not find container \"185e2ef0dd40865e0571096a6bc4651086b85b58f7b157a35d9c15b3e231d0cb\": container with ID starting with 185e2ef0dd40865e0571096a6bc4651086b85b58f7b157a35d9c15b3e231d0cb not found: ID does not exist" Jul 05 13:52:23.566678 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.566651 2567 scope.go:117] "RemoveContainer" containerID="bcd3265beaeed47e07cb478f21b1daeecc97249dee4bd91fcc51c833dd4f7a4c" Jul 05 13:52:23.566964 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:52:23.566944 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcd3265beaeed47e07cb478f21b1daeecc97249dee4bd91fcc51c833dd4f7a4c\": container with ID starting with bcd3265beaeed47e07cb478f21b1daeecc97249dee4bd91fcc51c833dd4f7a4c not found: ID does not exist" containerID="bcd3265beaeed47e07cb478f21b1daeecc97249dee4bd91fcc51c833dd4f7a4c" Jul 05 13:52:23.567012 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.566970 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcd3265beaeed47e07cb478f21b1daeecc97249dee4bd91fcc51c833dd4f7a4c"} err="failed to get container status \"bcd3265beaeed47e07cb478f21b1daeecc97249dee4bd91fcc51c833dd4f7a4c\": rpc error: code = NotFound desc = could not find container \"bcd3265beaeed47e07cb478f21b1daeecc97249dee4bd91fcc51c833dd4f7a4c\": container with ID starting with bcd3265beaeed47e07cb478f21b1daeecc97249dee4bd91fcc51c833dd4f7a4c not found: ID does not exist" Jul 05 13:52:23.571984 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.571955 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd"] Jul 05 13:52:23.578556 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:23.578528 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-76289d-openshift-default-84976548df-67wwd"] Jul 05 13:52:24.216175 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:24.216140 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3370c679-bd5a-49c7-bd54-5967cd8c9da1" path="/var/lib/kubelet/pods/3370c679-bd5a-49c7-bd54-5967cd8c9da1/volumes" Jul 05 13:52:26.018232 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:26.018192 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv"] Jul 05 13:52:26.018688 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:26.018421 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" podUID="c7b41d70-7920-4dc4-aea3-44bc85a4480a" containerName="istio-proxy" containerID="cri-o://8ba918ca06a83b26925419b6e880aaec845c9fab1ffbc5645b3c5b3f3d825d16" gracePeriod=30 Jul 05 13:52:26.875342 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:26.875309 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv"] Jul 05 13:52:26.875821 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:26.875807 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3370c679-bd5a-49c7-bd54-5967cd8c9da1" containerName="istio-proxy" Jul 05 13:52:26.875821 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:26.875823 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3370c679-bd5a-49c7-bd54-5967cd8c9da1" containerName="istio-proxy" Jul 05 13:52:26.875928 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:26.875853 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3370c679-bd5a-49c7-bd54-5967cd8c9da1" containerName="istio-proxy" Jul 05 13:52:26.875928 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:26.875859 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3370c679-bd5a-49c7-bd54-5967cd8c9da1" containerName="istio-proxy" Jul 05 13:52:26.875991 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:26.875929 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3370c679-bd5a-49c7-bd54-5967cd8c9da1" containerName="istio-proxy" Jul 05 13:52:26.875991 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:26.875936 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3370c679-bd5a-49c7-bd54-5967cd8c9da1" containerName="istio-proxy" Jul 05 13:52:26.880627 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:26.880596 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:26.883766 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:26.883713 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-4fa5fb-openshift-default-dockercfg-fdkdq\"" Jul 05 13:52:26.893957 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:26.893925 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv"] Jul 05 13:52:26.963731 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:26.963686 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istio-podinfo\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:26.963959 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:26.963906 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-workload-socket\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:26.964036 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:26.963973 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istio-envoy\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:26.964036 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:26.963996 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istio-token\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:26.964182 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:26.964143 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-workload-certs\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:26.964261 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:26.964225 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istio-data\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:26.964261 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:26.964253 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-265g2\" (UniqueName: \"kubernetes.io/projected/96a1c209-b3f1-4de0-9b74-9c2273e51d80-kube-api-access-265g2\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:26.964475 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:26.964453 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-credential-socket\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:26.964564 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:26.964505 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istiod-ca-cert\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:27.065987 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.065947 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-credential-socket\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:27.065987 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.065993 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istiod-ca-cert\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:27.066531 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.066025 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istio-podinfo\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:27.066531 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.066055 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-workload-socket\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:27.066531 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.066087 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istio-envoy\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:27.066531 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.066104 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istio-token\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:27.066531 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.066127 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-workload-certs\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:27.066531 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.066266 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istio-data\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:27.066531 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.066343 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-265g2\" (UniqueName: \"kubernetes.io/projected/96a1c209-b3f1-4de0-9b74-9c2273e51d80-kube-api-access-265g2\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:27.066531 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.066492 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-credential-socket\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:27.067047 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.066635 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-workload-socket\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:27.067047 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.066695 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istio-data\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:27.067047 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.066836 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istiod-ca-cert\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:27.067047 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.066921 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-workload-certs\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:27.068656 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.068634 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istio-envoy\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:27.068900 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.068881 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istio-podinfo\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:27.077397 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.077364 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istio-token\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:27.077581 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.077556 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-265g2\" (UniqueName: \"kubernetes.io/projected/96a1c209-b3f1-4de0-9b74-9c2273e51d80-kube-api-access-265g2\") pod \"e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:27.194024 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.193981 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:27.366790 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.366758 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv"] Jul 05 13:52:27.368618 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:52:27.368587 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96a1c209_b3f1_4de0_9b74_9c2273e51d80.slice/crio-2bccbe0ce1ac908b91ac11eb0219c62dcc3aa2d392724cd764af54dc611df5e9 WatchSource:0}: Error finding container 2bccbe0ce1ac908b91ac11eb0219c62dcc3aa2d392724cd764af54dc611df5e9: Status 404 returned error can't find the container with id 2bccbe0ce1ac908b91ac11eb0219c62dcc3aa2d392724cd764af54dc611df5e9 Jul 05 13:52:27.370947 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.370914 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:52:27.371063 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.370987 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:52:27.371063 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.371017 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:52:27.575817 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.575710 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" event={"ID":"96a1c209-b3f1-4de0-9b74-9c2273e51d80","Type":"ContainerStarted","Data":"e92a5df58f59b3e54e22ff6a75429e18e25d1dd6e00b5a406d1994d8cbcc44a5"} Jul 05 13:52:27.575817 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.575795 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" event={"ID":"96a1c209-b3f1-4de0-9b74-9c2273e51d80","Type":"ContainerStarted","Data":"2bccbe0ce1ac908b91ac11eb0219c62dcc3aa2d392724cd764af54dc611df5e9"} Jul 05 13:52:27.603486 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:27.603418 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" podStartSLOduration=1.6033936629999999 podStartE2EDuration="1.603393663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:52:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:52:27.600460874 +0000 UTC m=+2033.941478646" watchObservedRunningTime="2026-07-05 13:52:27.603393663 +0000 UTC m=+2033.944411436" Jul 05 13:52:28.194500 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:28.194455 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:28.199616 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:28.199592 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:28.248562 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:28.248526 2567 patch_prober.go:28] interesting pod/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 05 13:52:28.248719 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:28.248599 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" podUID="c7b41d70-7920-4dc4-aea3-44bc85a4480a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 05 13:52:28.581581 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:28.581477 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:28.582665 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:28.582643 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:31.287292 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.287207 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:52:31.413207 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.413164 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istiod-ca-cert\") pod \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " Jul 05 13:52:31.413384 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.413230 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istio-podinfo\") pod \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " Jul 05 13:52:31.413384 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.413292 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-workload-socket\") pod \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " Jul 05 13:52:31.413384 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.413323 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-workload-certs\") pod \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " Jul 05 13:52:31.413384 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.413371 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-credential-socket\") pod \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " Jul 05 13:52:31.413592 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.413387 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istio-data\") pod \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " Jul 05 13:52:31.413592 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.413402 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istio-token\") pod \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " Jul 05 13:52:31.413592 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.413430 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfp6b\" (UniqueName: \"kubernetes.io/projected/c7b41d70-7920-4dc4-aea3-44bc85a4480a-kube-api-access-sfp6b\") pod \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " Jul 05 13:52:31.413592 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.413494 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istio-envoy\") pod \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\" (UID: \"c7b41d70-7920-4dc4-aea3-44bc85a4480a\") " Jul 05 13:52:31.413836 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.413595 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c7b41d70-7920-4dc4-aea3-44bc85a4480a" (UID: "c7b41d70-7920-4dc4-aea3-44bc85a4480a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:52:31.413836 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.413646 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c7b41d70-7920-4dc4-aea3-44bc85a4480a" (UID: "c7b41d70-7920-4dc4-aea3-44bc85a4480a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:52:31.413836 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.413651 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c7b41d70-7920-4dc4-aea3-44bc85a4480a" (UID: "c7b41d70-7920-4dc4-aea3-44bc85a4480a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:52:31.414006 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.413985 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-workload-socket\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:52:31.414068 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.414012 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-workload-certs\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:52:31.414068 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.414032 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istiod-ca-cert\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:52:31.414338 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.414060 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c7b41d70-7920-4dc4-aea3-44bc85a4480a" (UID: "c7b41d70-7920-4dc4-aea3-44bc85a4480a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:52:31.414729 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.414702 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istio-data" (OuterVolumeSpecName: "istio-data") pod "c7b41d70-7920-4dc4-aea3-44bc85a4480a" (UID: "c7b41d70-7920-4dc4-aea3-44bc85a4480a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:52:31.415784 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.415760 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c7b41d70-7920-4dc4-aea3-44bc85a4480a" (UID: "c7b41d70-7920-4dc4-aea3-44bc85a4480a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 05 13:52:31.415933 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.415908 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7b41d70-7920-4dc4-aea3-44bc85a4480a-kube-api-access-sfp6b" (OuterVolumeSpecName: "kube-api-access-sfp6b") pod "c7b41d70-7920-4dc4-aea3-44bc85a4480a" (UID: "c7b41d70-7920-4dc4-aea3-44bc85a4480a"). InnerVolumeSpecName "kube-api-access-sfp6b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:52:31.416141 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.416118 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istio-token" (OuterVolumeSpecName: "istio-token") pod "c7b41d70-7920-4dc4-aea3-44bc85a4480a" (UID: "c7b41d70-7920-4dc4-aea3-44bc85a4480a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:52:31.416255 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.416148 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c7b41d70-7920-4dc4-aea3-44bc85a4480a" (UID: "c7b41d70-7920-4dc4-aea3-44bc85a4480a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:52:31.515486 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.515438 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-credential-socket\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:52:31.515486 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.515477 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istio-data\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:52:31.515486 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.515487 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istio-token\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:52:31.515486 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.515495 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfp6b\" (UniqueName: \"kubernetes.io/projected/c7b41d70-7920-4dc4-aea3-44bc85a4480a-kube-api-access-sfp6b\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:52:31.515810 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.515505 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istio-envoy\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:52:31.515810 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.515513 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7b41d70-7920-4dc4-aea3-44bc85a4480a-istio-podinfo\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:52:31.604159 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.604059 2567 generic.go:356] "Generic (PLEG): container finished" podID="c7b41d70-7920-4dc4-aea3-44bc85a4480a" containerID="8ba918ca06a83b26925419b6e880aaec845c9fab1ffbc5645b3c5b3f3d825d16" exitCode=0 Jul 05 13:52:31.604159 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.604108 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" Jul 05 13:52:31.604159 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.604136 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" event={"ID":"c7b41d70-7920-4dc4-aea3-44bc85a4480a","Type":"ContainerDied","Data":"8ba918ca06a83b26925419b6e880aaec845c9fab1ffbc5645b3c5b3f3d825d16"} Jul 05 13:52:31.604431 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.604194 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv" event={"ID":"c7b41d70-7920-4dc4-aea3-44bc85a4480a","Type":"ContainerDied","Data":"c65ffad8af7c22e0fb67eec3540820a4e41a1e255e6b5f7279f14cf8ca9da8cf"} Jul 05 13:52:31.604431 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.604216 2567 scope.go:117] "RemoveContainer" containerID="8ba918ca06a83b26925419b6e880aaec845c9fab1ffbc5645b3c5b3f3d825d16" Jul 05 13:52:31.614410 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.614384 2567 scope.go:117] "RemoveContainer" containerID="ef10f405be214ff64d158747387240f5b76b10af052e704ae03868bab4fb85ca" Jul 05 13:52:31.623367 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.623343 2567 scope.go:117] "RemoveContainer" containerID="8ba918ca06a83b26925419b6e880aaec845c9fab1ffbc5645b3c5b3f3d825d16" Jul 05 13:52:31.623665 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:52:31.623646 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ba918ca06a83b26925419b6e880aaec845c9fab1ffbc5645b3c5b3f3d825d16\": container with ID starting with 8ba918ca06a83b26925419b6e880aaec845c9fab1ffbc5645b3c5b3f3d825d16 not found: ID does not exist" containerID="8ba918ca06a83b26925419b6e880aaec845c9fab1ffbc5645b3c5b3f3d825d16" Jul 05 13:52:31.623708 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.623677 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ba918ca06a83b26925419b6e880aaec845c9fab1ffbc5645b3c5b3f3d825d16"} err="failed to get container status \"8ba918ca06a83b26925419b6e880aaec845c9fab1ffbc5645b3c5b3f3d825d16\": rpc error: code = NotFound desc = could not find container \"8ba918ca06a83b26925419b6e880aaec845c9fab1ffbc5645b3c5b3f3d825d16\": container with ID starting with 8ba918ca06a83b26925419b6e880aaec845c9fab1ffbc5645b3c5b3f3d825d16 not found: ID does not exist" Jul 05 13:52:31.623708 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.623696 2567 scope.go:117] "RemoveContainer" containerID="ef10f405be214ff64d158747387240f5b76b10af052e704ae03868bab4fb85ca" Jul 05 13:52:31.624031 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:52:31.624009 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef10f405be214ff64d158747387240f5b76b10af052e704ae03868bab4fb85ca\": container with ID starting with ef10f405be214ff64d158747387240f5b76b10af052e704ae03868bab4fb85ca not found: ID does not exist" containerID="ef10f405be214ff64d158747387240f5b76b10af052e704ae03868bab4fb85ca" Jul 05 13:52:31.624065 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.624039 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef10f405be214ff64d158747387240f5b76b10af052e704ae03868bab4fb85ca"} err="failed to get container status \"ef10f405be214ff64d158747387240f5b76b10af052e704ae03868bab4fb85ca\": rpc error: code = NotFound desc = could not find container \"ef10f405be214ff64d158747387240f5b76b10af052e704ae03868bab4fb85ca\": container with ID starting with ef10f405be214ff64d158747387240f5b76b10af052e704ae03868bab4fb85ca not found: ID does not exist" Jul 05 13:52:31.627765 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.627723 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv"] Jul 05 13:52:31.633202 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:31.633176 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-4b5487-openshift-default-57ccccc79-55stv"] Jul 05 13:52:32.216962 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:32.216929 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7b41d70-7920-4dc4-aea3-44bc85a4480a" path="/var/lib/kubelet/pods/c7b41d70-7920-4dc4-aea3-44bc85a4480a/volumes" Jul 05 13:52:34.808529 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:34.808435 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw"] Jul 05 13:52:34.809147 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:34.809127 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7b41d70-7920-4dc4-aea3-44bc85a4480a" containerName="istio-proxy" Jul 05 13:52:34.809234 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:34.809150 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7b41d70-7920-4dc4-aea3-44bc85a4480a" containerName="istio-proxy" Jul 05 13:52:34.809306 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:34.809290 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7b41d70-7920-4dc4-aea3-44bc85a4480a" containerName="istio-proxy" Jul 05 13:52:34.809360 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:34.809315 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7b41d70-7920-4dc4-aea3-44bc85a4480a" containerName="istio-proxy" Jul 05 13:52:34.814071 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:34.814047 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:34.817300 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:34.817272 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-ed2315-openshift-default-dockercfg-2mtj8\"" Jul 05 13:52:34.832006 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:34.831973 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw"] Jul 05 13:52:34.950286 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:34.950242 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-workload-certs\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:34.950286 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:34.950286 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a25b8d1d-836b-4a9a-851c-892193b21333-istiod-ca-cert\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:34.950515 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:34.950377 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-istio-envoy\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:34.950515 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:34.950426 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a25b8d1d-836b-4a9a-851c-892193b21333-istio-podinfo\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:34.950515 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:34.950498 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-credential-socket\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:34.950612 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:34.950535 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a25b8d1d-836b-4a9a-851c-892193b21333-istio-token\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:34.950646 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:34.950630 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4tjb\" (UniqueName: \"kubernetes.io/projected/a25b8d1d-836b-4a9a-851c-892193b21333-kube-api-access-p4tjb\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:34.950684 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:34.950662 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-istio-data\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:34.950719 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:34.950686 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-workload-socket\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:35.051580 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.051539 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-workload-certs\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:35.051580 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.051580 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a25b8d1d-836b-4a9a-851c-892193b21333-istiod-ca-cert\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:35.051861 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.051624 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-istio-envoy\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:35.051861 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.051666 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a25b8d1d-836b-4a9a-851c-892193b21333-istio-podinfo\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:35.051861 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.051712 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-credential-socket\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:35.051861 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.051768 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a25b8d1d-836b-4a9a-851c-892193b21333-istio-token\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:35.052048 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.051860 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4tjb\" (UniqueName: \"kubernetes.io/projected/a25b8d1d-836b-4a9a-851c-892193b21333-kube-api-access-p4tjb\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:35.052048 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.051893 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-istio-data\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:35.052048 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.051924 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-workload-socket\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:35.052048 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.052021 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-workload-certs\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:35.052248 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.052076 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-credential-socket\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:35.052370 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.052323 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-workload-socket\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:35.052370 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.052363 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-istio-data\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:35.052688 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.052651 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a25b8d1d-836b-4a9a-851c-892193b21333-istiod-ca-cert\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:35.054323 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.054293 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-istio-envoy\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:35.054486 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.054465 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a25b8d1d-836b-4a9a-851c-892193b21333-istio-podinfo\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:35.066238 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.066173 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4tjb\" (UniqueName: \"kubernetes.io/projected/a25b8d1d-836b-4a9a-851c-892193b21333-kube-api-access-p4tjb\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:35.079313 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.079284 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a25b8d1d-836b-4a9a-851c-892193b21333-istio-token\") pod \"e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:35.126291 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.126246 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:35.312717 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.312677 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw"] Jul 05 13:52:35.315305 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:52:35.315272 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda25b8d1d_836b_4a9a_851c_892193b21333.slice/crio-0f994a216eed31fb442ea2cd79944d30686aef6d63cadc324c94bf83a54f33bb WatchSource:0}: Error finding container 0f994a216eed31fb442ea2cd79944d30686aef6d63cadc324c94bf83a54f33bb: Status 404 returned error can't find the container with id 0f994a216eed31fb442ea2cd79944d30686aef6d63cadc324c94bf83a54f33bb Jul 05 13:52:35.317539 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.317506 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:52:35.317649 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.317587 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:52:35.317649 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.317632 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:52:35.634122 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.634022 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" event={"ID":"a25b8d1d-836b-4a9a-851c-892193b21333","Type":"ContainerStarted","Data":"168379253aa660ae5a14b3a462a279bea5418a79c60a39109d2029a85b4bbe27"} Jul 05 13:52:35.634122 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.634085 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" event={"ID":"a25b8d1d-836b-4a9a-851c-892193b21333","Type":"ContainerStarted","Data":"0f994a216eed31fb442ea2cd79944d30686aef6d63cadc324c94bf83a54f33bb"} Jul 05 13:52:35.659826 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:35.659770 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" podStartSLOduration=1.6597353209999999 podStartE2EDuration="1.659735321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:52:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:52:35.656777266 +0000 UTC m=+2041.997795043" watchObservedRunningTime="2026-07-05 13:52:35.659735321 +0000 UTC m=+2042.000753092" Jul 05 13:52:36.126887 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:36.126846 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:36.128635 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:36.128602 2567 patch_prober.go:28] interesting pod/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 05 13:52:36.128798 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:36.128658 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" podUID="a25b8d1d-836b-4a9a-851c-892193b21333" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 05 13:52:37.131172 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:37.131123 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:37.649812 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:37.649778 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:37.653960 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:37.653928 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:44.623946 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:44.623890 2567 patch_prober.go:28] interesting pod/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 05 13:52:44.624373 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:44.623984 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" podUID="96a1c209-b3f1-4de0-9b74-9c2273e51d80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 05 13:52:45.702116 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:45.702082 2567 generic.go:356] "Generic (PLEG): container finished" podID="96a1c209-b3f1-4de0-9b74-9c2273e51d80" containerID="e92a5df58f59b3e54e22ff6a75429e18e25d1dd6e00b5a406d1994d8cbcc44a5" exitCode=0 Jul 05 13:52:45.702556 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:45.702156 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" event={"ID":"96a1c209-b3f1-4de0-9b74-9c2273e51d80","Type":"ContainerDied","Data":"e92a5df58f59b3e54e22ff6a75429e18e25d1dd6e00b5a406d1994d8cbcc44a5"} Jul 05 13:52:45.702611 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:45.702597 2567 scope.go:117] "RemoveContainer" containerID="e92a5df58f59b3e54e22ff6a75429e18e25d1dd6e00b5a406d1994d8cbcc44a5" Jul 05 13:52:45.703782 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:45.703757 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:52:45.703856 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:45.703831 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:52:45.703894 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:45.703862 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:52:46.709706 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:46.709668 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" event={"ID":"96a1c209-b3f1-4de0-9b74-9c2273e51d80","Type":"ContainerStarted","Data":"8db1f4d1b9a017a12a0f3a821fb4e6f548525e068934d6db67650517dc2fdf3c"} Jul 05 13:52:47.194921 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:47.194879 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:47.195154 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:47.194945 2567 patch_prober.go:28] interesting pod/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 05 13:52:47.195154 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:47.195024 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" podUID="96a1c209-b3f1-4de0-9b74-9c2273e51d80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 05 13:52:48.194859 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:48.194810 2567 patch_prober.go:28] interesting pod/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 05 13:52:48.195421 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:48.194884 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" podUID="96a1c209-b3f1-4de0-9b74-9c2273e51d80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 05 13:52:49.194869 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:49.194830 2567 patch_prober.go:28] interesting pod/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 05 13:52:49.195590 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:49.195555 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" podUID="96a1c209-b3f1-4de0-9b74-9c2273e51d80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 05 13:52:50.194781 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:50.194714 2567 patch_prober.go:28] interesting pod/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 05 13:52:50.195003 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:50.194831 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" podUID="96a1c209-b3f1-4de0-9b74-9c2273e51d80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 05 13:52:51.194954 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:51.194899 2567 patch_prober.go:28] interesting pod/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 05 13:52:51.195194 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:51.194988 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" podUID="96a1c209-b3f1-4de0-9b74-9c2273e51d80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 05 13:52:52.199577 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:52.199541 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:52.200065 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:52.199945 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:52.200872 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:52.200854 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:52:53.692958 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:53.692916 2567 patch_prober.go:28] interesting pod/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 05 13:52:53.693372 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:53.693008 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" podUID="a25b8d1d-836b-4a9a-851c-892193b21333" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 05 13:52:54.767519 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:54.767438 2567 generic.go:356] "Generic (PLEG): container finished" podID="a25b8d1d-836b-4a9a-851c-892193b21333" containerID="168379253aa660ae5a14b3a462a279bea5418a79c60a39109d2029a85b4bbe27" exitCode=0 Jul 05 13:52:54.768043 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:54.767540 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" event={"ID":"a25b8d1d-836b-4a9a-851c-892193b21333","Type":"ContainerDied","Data":"168379253aa660ae5a14b3a462a279bea5418a79c60a39109d2029a85b4bbe27"} Jul 05 13:52:54.768146 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:54.768128 2567 scope.go:117] "RemoveContainer" containerID="168379253aa660ae5a14b3a462a279bea5418a79c60a39109d2029a85b4bbe27" Jul 05 13:52:54.769397 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:54.769361 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:52:54.769528 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:54.769451 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:52:54.769528 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:54.769497 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:52:55.775976 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:55.775938 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" event={"ID":"a25b8d1d-836b-4a9a-851c-892193b21333","Type":"ContainerStarted","Data":"31d7f0153d3c2baf29f58a026d45059932fad64638c7fe0d19770295b0d840ea"} Jul 05 13:52:56.126807 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:56.126669 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:52:56.126990 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:56.126963 2567 patch_prober.go:28] interesting pod/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 05 13:52:56.127065 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:56.127030 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" podUID="a25b8d1d-836b-4a9a-851c-892193b21333" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 05 13:52:57.126858 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:57.126817 2567 patch_prober.go:28] interesting pod/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 05 13:52:57.127389 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:57.126883 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" podUID="a25b8d1d-836b-4a9a-851c-892193b21333" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 05 13:52:57.746010 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:57.745972 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv"] Jul 05 13:52:57.746281 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:57.746249 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" podUID="96a1c209-b3f1-4de0-9b74-9c2273e51d80" containerName="istio-proxy" containerID="cri-o://8db1f4d1b9a017a12a0f3a821fb4e6f548525e068934d6db67650517dc2fdf3c" gracePeriod=30 Jul 05 13:52:58.127041 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:58.126936 2567 patch_prober.go:28] interesting pod/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 05 13:52:58.127041 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:58.127017 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" podUID="a25b8d1d-836b-4a9a-851c-892193b21333" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 05 13:52:59.127128 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:59.127072 2567 patch_prober.go:28] interesting pod/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 05 13:52:59.127546 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:52:59.127158 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" podUID="a25b8d1d-836b-4a9a-851c-892193b21333" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 05 13:53:00.130473 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:00.130439 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:53:00.130974 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:00.130838 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:53:00.131447 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:00.131427 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:53:03.006650 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.006622 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:53:03.074169 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.074071 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-workload-certs\") pod \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " Jul 05 13:53:03.074169 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.074133 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istio-envoy\") pod \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " Jul 05 13:53:03.074169 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.074152 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istio-token\") pod \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " Jul 05 13:53:03.074453 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.074186 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-workload-socket\") pod \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " Jul 05 13:53:03.074453 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.074216 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istio-podinfo\") pod \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " Jul 05 13:53:03.074453 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.074234 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istio-data\") pod \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " Jul 05 13:53:03.074453 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.074267 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-credential-socket\") pod \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " Jul 05 13:53:03.074453 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.074319 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istiod-ca-cert\") pod \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " Jul 05 13:53:03.074453 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.074392 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-265g2\" (UniqueName: \"kubernetes.io/projected/96a1c209-b3f1-4de0-9b74-9c2273e51d80-kube-api-access-265g2\") pod \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\" (UID: \"96a1c209-b3f1-4de0-9b74-9c2273e51d80\") " Jul 05 13:53:03.074453 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.074438 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "96a1c209-b3f1-4de0-9b74-9c2273e51d80" (UID: "96a1c209-b3f1-4de0-9b74-9c2273e51d80"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:53:03.074862 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.074569 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "96a1c209-b3f1-4de0-9b74-9c2273e51d80" (UID: "96a1c209-b3f1-4de0-9b74-9c2273e51d80"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:53:03.074862 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.074668 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "96a1c209-b3f1-4de0-9b74-9c2273e51d80" (UID: "96a1c209-b3f1-4de0-9b74-9c2273e51d80"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:53:03.074974 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.074875 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "96a1c209-b3f1-4de0-9b74-9c2273e51d80" (UID: "96a1c209-b3f1-4de0-9b74-9c2273e51d80"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:53:03.074974 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.074894 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-credential-socket\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:53:03.074974 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.074913 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-workload-certs\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:53:03.074974 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.074927 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-workload-socket\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:53:03.075502 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.075472 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istio-data" (OuterVolumeSpecName: "istio-data") pod "96a1c209-b3f1-4de0-9b74-9c2273e51d80" (UID: "96a1c209-b3f1-4de0-9b74-9c2273e51d80"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:53:03.076667 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.076635 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istio-token" (OuterVolumeSpecName: "istio-token") pod "96a1c209-b3f1-4de0-9b74-9c2273e51d80" (UID: "96a1c209-b3f1-4de0-9b74-9c2273e51d80"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:53:03.076896 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.076876 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "96a1c209-b3f1-4de0-9b74-9c2273e51d80" (UID: "96a1c209-b3f1-4de0-9b74-9c2273e51d80"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 05 13:53:03.077160 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.077133 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "96a1c209-b3f1-4de0-9b74-9c2273e51d80" (UID: "96a1c209-b3f1-4de0-9b74-9c2273e51d80"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:53:03.077257 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.077172 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96a1c209-b3f1-4de0-9b74-9c2273e51d80-kube-api-access-265g2" (OuterVolumeSpecName: "kube-api-access-265g2") pod "96a1c209-b3f1-4de0-9b74-9c2273e51d80" (UID: "96a1c209-b3f1-4de0-9b74-9c2273e51d80"). InnerVolumeSpecName "kube-api-access-265g2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:53:03.176149 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.176115 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-265g2\" (UniqueName: \"kubernetes.io/projected/96a1c209-b3f1-4de0-9b74-9c2273e51d80-kube-api-access-265g2\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:53:03.176149 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.176145 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istio-envoy\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:53:03.176149 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.176155 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istio-token\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:53:03.176149 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.176163 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istio-podinfo\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:53:03.176436 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.176171 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istio-data\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:53:03.176436 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.176180 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96a1c209-b3f1-4de0-9b74-9c2273e51d80-istiod-ca-cert\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:53:03.838194 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.838159 2567 generic.go:356] "Generic (PLEG): container finished" podID="96a1c209-b3f1-4de0-9b74-9c2273e51d80" containerID="8db1f4d1b9a017a12a0f3a821fb4e6f548525e068934d6db67650517dc2fdf3c" exitCode=0 Jul 05 13:53:03.838403 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.838215 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" event={"ID":"96a1c209-b3f1-4de0-9b74-9c2273e51d80","Type":"ContainerDied","Data":"8db1f4d1b9a017a12a0f3a821fb4e6f548525e068934d6db67650517dc2fdf3c"} Jul 05 13:53:03.838403 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.838227 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" Jul 05 13:53:03.838403 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.838260 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv" event={"ID":"96a1c209-b3f1-4de0-9b74-9c2273e51d80","Type":"ContainerDied","Data":"2bccbe0ce1ac908b91ac11eb0219c62dcc3aa2d392724cd764af54dc611df5e9"} Jul 05 13:53:03.838403 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.838281 2567 scope.go:117] "RemoveContainer" containerID="8db1f4d1b9a017a12a0f3a821fb4e6f548525e068934d6db67650517dc2fdf3c" Jul 05 13:53:03.849074 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.849052 2567 scope.go:117] "RemoveContainer" containerID="e92a5df58f59b3e54e22ff6a75429e18e25d1dd6e00b5a406d1994d8cbcc44a5" Jul 05 13:53:03.858855 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.858483 2567 scope.go:117] "RemoveContainer" containerID="8db1f4d1b9a017a12a0f3a821fb4e6f548525e068934d6db67650517dc2fdf3c" Jul 05 13:53:03.858962 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:53:03.858874 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8db1f4d1b9a017a12a0f3a821fb4e6f548525e068934d6db67650517dc2fdf3c\": container with ID starting with 8db1f4d1b9a017a12a0f3a821fb4e6f548525e068934d6db67650517dc2fdf3c not found: ID does not exist" containerID="8db1f4d1b9a017a12a0f3a821fb4e6f548525e068934d6db67650517dc2fdf3c" Jul 05 13:53:03.858962 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.858913 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8db1f4d1b9a017a12a0f3a821fb4e6f548525e068934d6db67650517dc2fdf3c"} err="failed to get container status \"8db1f4d1b9a017a12a0f3a821fb4e6f548525e068934d6db67650517dc2fdf3c\": rpc error: code = NotFound desc = could not find container \"8db1f4d1b9a017a12a0f3a821fb4e6f548525e068934d6db67650517dc2fdf3c\": container with ID starting with 8db1f4d1b9a017a12a0f3a821fb4e6f548525e068934d6db67650517dc2fdf3c not found: ID does not exist" Jul 05 13:53:03.858962 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.858939 2567 scope.go:117] "RemoveContainer" containerID="e92a5df58f59b3e54e22ff6a75429e18e25d1dd6e00b5a406d1994d8cbcc44a5" Jul 05 13:53:03.859441 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:53:03.859261 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e92a5df58f59b3e54e22ff6a75429e18e25d1dd6e00b5a406d1994d8cbcc44a5\": container with ID starting with e92a5df58f59b3e54e22ff6a75429e18e25d1dd6e00b5a406d1994d8cbcc44a5 not found: ID does not exist" containerID="e92a5df58f59b3e54e22ff6a75429e18e25d1dd6e00b5a406d1994d8cbcc44a5" Jul 05 13:53:03.859441 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.859297 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e92a5df58f59b3e54e22ff6a75429e18e25d1dd6e00b5a406d1994d8cbcc44a5"} err="failed to get container status \"e92a5df58f59b3e54e22ff6a75429e18e25d1dd6e00b5a406d1994d8cbcc44a5\": rpc error: code = NotFound desc = could not find container \"e92a5df58f59b3e54e22ff6a75429e18e25d1dd6e00b5a406d1994d8cbcc44a5\": container with ID starting with e92a5df58f59b3e54e22ff6a75429e18e25d1dd6e00b5a406d1994d8cbcc44a5 not found: ID does not exist" Jul 05 13:53:03.861648 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.861619 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv"] Jul 05 13:53:03.867309 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:03.867271 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-4fa5fb-openshift-default-6dbbf76b59-78tcv"] Jul 05 13:53:04.216402 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:04.216367 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96a1c209-b3f1-4de0-9b74-9c2273e51d80" path="/var/lib/kubelet/pods/96a1c209-b3f1-4de0-9b74-9c2273e51d80/volumes" Jul 05 13:53:05.077958 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:05.077920 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw"] Jul 05 13:53:05.078192 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:05.078163 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" podUID="a25b8d1d-836b-4a9a-851c-892193b21333" containerName="istio-proxy" containerID="cri-o://31d7f0153d3c2baf29f58a026d45059932fad64638c7fe0d19770295b0d840ea" gracePeriod=30 Jul 05 13:53:06.000508 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.000467 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq"] Jul 05 13:53:06.001008 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.000994 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7b41d70-7920-4dc4-aea3-44bc85a4480a" containerName="istio-proxy" Jul 05 13:53:06.001077 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.001011 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7b41d70-7920-4dc4-aea3-44bc85a4480a" containerName="istio-proxy" Jul 05 13:53:06.001077 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.001029 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96a1c209-b3f1-4de0-9b74-9c2273e51d80" containerName="istio-proxy" Jul 05 13:53:06.001077 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.001034 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="96a1c209-b3f1-4de0-9b74-9c2273e51d80" containerName="istio-proxy" Jul 05 13:53:06.001077 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.001041 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96a1c209-b3f1-4de0-9b74-9c2273e51d80" containerName="istio-proxy" Jul 05 13:53:06.001077 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.001047 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="96a1c209-b3f1-4de0-9b74-9c2273e51d80" containerName="istio-proxy" Jul 05 13:53:06.001245 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.001137 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="96a1c209-b3f1-4de0-9b74-9c2273e51d80" containerName="istio-proxy" Jul 05 13:53:06.001245 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.001144 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="96a1c209-b3f1-4de0-9b74-9c2273e51d80" containerName="istio-proxy" Jul 05 13:53:06.012269 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.012240 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.015461 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.015422 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-6096ec-openshift-default-dockercfg-7s97g\"" Jul 05 13:53:06.019265 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.019225 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq"] Jul 05 13:53:06.110667 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.110622 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3416d16d-35d4-4b1d-8065-0013e0727b0c-istio-token\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.110925 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.110793 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-istio-data\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.110925 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.110831 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3416d16d-35d4-4b1d-8065-0013e0727b0c-istio-podinfo\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.110925 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.110854 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3416d16d-35d4-4b1d-8065-0013e0727b0c-istiod-ca-cert\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.111111 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.110931 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzbww\" (UniqueName: \"kubernetes.io/projected/3416d16d-35d4-4b1d-8065-0013e0727b0c-kube-api-access-wzbww\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.111111 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.110959 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-credential-socket\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.111111 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.111009 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-workload-socket\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.111111 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.111054 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-istio-envoy\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.111268 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.111109 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-workload-certs\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.211973 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.211940 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-workload-certs\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.212165 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.211989 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3416d16d-35d4-4b1d-8065-0013e0727b0c-istio-token\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.212165 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.212049 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-istio-data\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.212165 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.212072 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3416d16d-35d4-4b1d-8065-0013e0727b0c-istio-podinfo\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.212165 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.212094 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3416d16d-35d4-4b1d-8065-0013e0727b0c-istiod-ca-cert\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.212165 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.212158 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzbww\" (UniqueName: \"kubernetes.io/projected/3416d16d-35d4-4b1d-8065-0013e0727b0c-kube-api-access-wzbww\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.212476 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.212182 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-credential-socket\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.212476 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.212232 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-workload-socket\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.212476 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.212263 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-istio-envoy\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.212476 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.212464 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-workload-certs\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.212688 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.212559 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-istio-data\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.212688 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.212617 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-credential-socket\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.212912 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.212893 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-workload-socket\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.212979 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.212934 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3416d16d-35d4-4b1d-8065-0013e0727b0c-istiod-ca-cert\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.214807 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.214783 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-istio-envoy\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.215293 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.215268 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3416d16d-35d4-4b1d-8065-0013e0727b0c-istio-podinfo\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.220981 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.220954 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3416d16d-35d4-4b1d-8065-0013e0727b0c-istio-token\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.221259 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.221240 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzbww\" (UniqueName: \"kubernetes.io/projected/3416d16d-35d4-4b1d-8065-0013e0727b0c-kube-api-access-wzbww\") pod \"e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.326413 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.326307 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:06.490061 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.490026 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq"] Jul 05 13:53:06.491225 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:53:06.491189 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3416d16d_35d4_4b1d_8065_0013e0727b0c.slice/crio-f321d850a2b9c2ad899b48a2d4f1eda5b1d5ad0c19687871bf7d3a637a653d73 WatchSource:0}: Error finding container f321d850a2b9c2ad899b48a2d4f1eda5b1d5ad0c19687871bf7d3a637a653d73: Status 404 returned error can't find the container with id f321d850a2b9c2ad899b48a2d4f1eda5b1d5ad0c19687871bf7d3a637a653d73 Jul 05 13:53:06.493837 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.493795 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:53:06.493963 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.493884 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:53:06.493963 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.493927 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:53:06.859684 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.859641 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" event={"ID":"3416d16d-35d4-4b1d-8065-0013e0727b0c","Type":"ContainerStarted","Data":"8f06c2e9feafd22f60ad4cb61bf99d2895e95002b3aa931b358ef540ce188149"} Jul 05 13:53:06.859950 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.859701 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" event={"ID":"3416d16d-35d4-4b1d-8065-0013e0727b0c","Type":"ContainerStarted","Data":"f321d850a2b9c2ad899b48a2d4f1eda5b1d5ad0c19687871bf7d3a637a653d73"} Jul 05 13:53:06.888870 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:06.888777 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" podStartSLOduration=1.888734516 podStartE2EDuration="1.888734516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:53:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:53:06.886952583 +0000 UTC m=+2073.227970356" watchObservedRunningTime="2026-07-05 13:53:06.888734516 +0000 UTC m=+2073.229752290" Jul 05 13:53:07.327456 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:07.327425 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:07.332637 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:07.332595 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:07.865547 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:07.865510 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:07.866731 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:07.866709 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:10.358633 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.358604 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:53:10.455821 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.455783 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-workload-certs\") pod \"a25b8d1d-836b-4a9a-851c-892193b21333\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " Jul 05 13:53:10.456002 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.455868 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4tjb\" (UniqueName: \"kubernetes.io/projected/a25b8d1d-836b-4a9a-851c-892193b21333-kube-api-access-p4tjb\") pod \"a25b8d1d-836b-4a9a-851c-892193b21333\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " Jul 05 13:53:10.456002 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.455918 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-credential-socket\") pod \"a25b8d1d-836b-4a9a-851c-892193b21333\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " Jul 05 13:53:10.456002 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.455981 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-workload-socket\") pod \"a25b8d1d-836b-4a9a-851c-892193b21333\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " Jul 05 13:53:10.456144 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.456014 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a25b8d1d-836b-4a9a-851c-892193b21333-istio-token\") pod \"a25b8d1d-836b-4a9a-851c-892193b21333\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " Jul 05 13:53:10.456144 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.456064 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-istio-data\") pod \"a25b8d1d-836b-4a9a-851c-892193b21333\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " Jul 05 13:53:10.456144 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.456096 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a25b8d1d-836b-4a9a-851c-892193b21333-istiod-ca-cert\") pod \"a25b8d1d-836b-4a9a-851c-892193b21333\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " Jul 05 13:53:10.456144 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.456113 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a25b8d1d-836b-4a9a-851c-892193b21333" (UID: "a25b8d1d-836b-4a9a-851c-892193b21333"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:53:10.456364 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.456224 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a25b8d1d-836b-4a9a-851c-892193b21333" (UID: "a25b8d1d-836b-4a9a-851c-892193b21333"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:53:10.456364 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.456315 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a25b8d1d-836b-4a9a-851c-892193b21333" (UID: "a25b8d1d-836b-4a9a-851c-892193b21333"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:53:10.456537 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.456494 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a25b8d1d-836b-4a9a-851c-892193b21333-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a25b8d1d-836b-4a9a-851c-892193b21333" (UID: "a25b8d1d-836b-4a9a-851c-892193b21333"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:53:10.456537 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.456533 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a25b8d1d-836b-4a9a-851c-892193b21333-istio-podinfo\") pod \"a25b8d1d-836b-4a9a-851c-892193b21333\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " Jul 05 13:53:10.456646 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.456594 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-istio-envoy\") pod \"a25b8d1d-836b-4a9a-851c-892193b21333\" (UID: \"a25b8d1d-836b-4a9a-851c-892193b21333\") " Jul 05 13:53:10.457145 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.457111 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-istio-data" (OuterVolumeSpecName: "istio-data") pod "a25b8d1d-836b-4a9a-851c-892193b21333" (UID: "a25b8d1d-836b-4a9a-851c-892193b21333"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:53:10.457533 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.457505 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-workload-socket\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:53:10.457653 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.457538 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-istio-data\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:53:10.457653 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.457552 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a25b8d1d-836b-4a9a-851c-892193b21333-istiod-ca-cert\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:53:10.457653 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.457566 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-workload-certs\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:53:10.457653 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.457579 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-credential-socket\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:53:10.458573 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.458544 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a25b8d1d-836b-4a9a-851c-892193b21333-kube-api-access-p4tjb" (OuterVolumeSpecName: "kube-api-access-p4tjb") pod "a25b8d1d-836b-4a9a-851c-892193b21333" (UID: "a25b8d1d-836b-4a9a-851c-892193b21333"). InnerVolumeSpecName "kube-api-access-p4tjb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:53:10.458668 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.458638 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a25b8d1d-836b-4a9a-851c-892193b21333-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a25b8d1d-836b-4a9a-851c-892193b21333" (UID: "a25b8d1d-836b-4a9a-851c-892193b21333"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 05 13:53:10.459082 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.459061 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a25b8d1d-836b-4a9a-851c-892193b21333-istio-token" (OuterVolumeSpecName: "istio-token") pod "a25b8d1d-836b-4a9a-851c-892193b21333" (UID: "a25b8d1d-836b-4a9a-851c-892193b21333"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:53:10.459349 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.459327 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a25b8d1d-836b-4a9a-851c-892193b21333" (UID: "a25b8d1d-836b-4a9a-851c-892193b21333"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:53:10.558400 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.558359 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a25b8d1d-836b-4a9a-851c-892193b21333-istio-token\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:53:10.558400 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.558395 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a25b8d1d-836b-4a9a-851c-892193b21333-istio-podinfo\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:53:10.558400 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.558404 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a25b8d1d-836b-4a9a-851c-892193b21333-istio-envoy\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:53:10.558649 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.558415 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4tjb\" (UniqueName: \"kubernetes.io/projected/a25b8d1d-836b-4a9a-851c-892193b21333-kube-api-access-p4tjb\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:53:10.890608 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.890574 2567 generic.go:356] "Generic (PLEG): container finished" podID="a25b8d1d-836b-4a9a-851c-892193b21333" containerID="31d7f0153d3c2baf29f58a026d45059932fad64638c7fe0d19770295b0d840ea" exitCode=0 Jul 05 13:53:10.890829 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.890636 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" Jul 05 13:53:10.890829 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.890664 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" event={"ID":"a25b8d1d-836b-4a9a-851c-892193b21333","Type":"ContainerDied","Data":"31d7f0153d3c2baf29f58a026d45059932fad64638c7fe0d19770295b0d840ea"} Jul 05 13:53:10.890829 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.890735 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw" event={"ID":"a25b8d1d-836b-4a9a-851c-892193b21333","Type":"ContainerDied","Data":"0f994a216eed31fb442ea2cd79944d30686aef6d63cadc324c94bf83a54f33bb"} Jul 05 13:53:10.890829 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.890781 2567 scope.go:117] "RemoveContainer" containerID="31d7f0153d3c2baf29f58a026d45059932fad64638c7fe0d19770295b0d840ea" Jul 05 13:53:10.911720 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.911692 2567 scope.go:117] "RemoveContainer" containerID="168379253aa660ae5a14b3a462a279bea5418a79c60a39109d2029a85b4bbe27" Jul 05 13:53:10.921603 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.921547 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw"] Jul 05 13:53:10.921665 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.921635 2567 scope.go:117] "RemoveContainer" containerID="31d7f0153d3c2baf29f58a026d45059932fad64638c7fe0d19770295b0d840ea" Jul 05 13:53:10.921977 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:53:10.921952 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31d7f0153d3c2baf29f58a026d45059932fad64638c7fe0d19770295b0d840ea\": container with ID starting with 31d7f0153d3c2baf29f58a026d45059932fad64638c7fe0d19770295b0d840ea not found: ID does not exist" containerID="31d7f0153d3c2baf29f58a026d45059932fad64638c7fe0d19770295b0d840ea" Jul 05 13:53:10.922051 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.921984 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31d7f0153d3c2baf29f58a026d45059932fad64638c7fe0d19770295b0d840ea"} err="failed to get container status \"31d7f0153d3c2baf29f58a026d45059932fad64638c7fe0d19770295b0d840ea\": rpc error: code = NotFound desc = could not find container \"31d7f0153d3c2baf29f58a026d45059932fad64638c7fe0d19770295b0d840ea\": container with ID starting with 31d7f0153d3c2baf29f58a026d45059932fad64638c7fe0d19770295b0d840ea not found: ID does not exist" Jul 05 13:53:10.922051 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.922005 2567 scope.go:117] "RemoveContainer" containerID="168379253aa660ae5a14b3a462a279bea5418a79c60a39109d2029a85b4bbe27" Jul 05 13:53:10.922445 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:53:10.922412 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"168379253aa660ae5a14b3a462a279bea5418a79c60a39109d2029a85b4bbe27\": container with ID starting with 168379253aa660ae5a14b3a462a279bea5418a79c60a39109d2029a85b4bbe27 not found: ID does not exist" containerID="168379253aa660ae5a14b3a462a279bea5418a79c60a39109d2029a85b4bbe27" Jul 05 13:53:10.922575 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.922453 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"168379253aa660ae5a14b3a462a279bea5418a79c60a39109d2029a85b4bbe27"} err="failed to get container status \"168379253aa660ae5a14b3a462a279bea5418a79c60a39109d2029a85b4bbe27\": rpc error: code = NotFound desc = could not find container \"168379253aa660ae5a14b3a462a279bea5418a79c60a39109d2029a85b4bbe27\": container with ID starting with 168379253aa660ae5a14b3a462a279bea5418a79c60a39109d2029a85b4bbe27 not found: ID does not exist" Jul 05 13:53:10.925356 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:10.925328 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-ed2315-openshift-default-674f856fd5-9ltzw"] Jul 05 13:53:12.217644 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:12.217611 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a25b8d1d-836b-4a9a-851c-892193b21333" path="/var/lib/kubelet/pods/a25b8d1d-836b-4a9a-851c-892193b21333/volumes" Jul 05 13:53:14.008135 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.008094 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh"] Jul 05 13:53:14.008637 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.008621 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a25b8d1d-836b-4a9a-851c-892193b21333" containerName="istio-proxy" Jul 05 13:53:14.008637 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.008638 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a25b8d1d-836b-4a9a-851c-892193b21333" containerName="istio-proxy" Jul 05 13:53:14.008724 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.008653 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a25b8d1d-836b-4a9a-851c-892193b21333" containerName="istio-proxy" Jul 05 13:53:14.008724 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.008659 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a25b8d1d-836b-4a9a-851c-892193b21333" containerName="istio-proxy" Jul 05 13:53:14.008810 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.008778 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a25b8d1d-836b-4a9a-851c-892193b21333" containerName="istio-proxy" Jul 05 13:53:14.008810 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.008788 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a25b8d1d-836b-4a9a-851c-892193b21333" containerName="istio-proxy" Jul 05 13:53:14.013632 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.013603 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.017799 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.017775 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-6579e1-openshift-default-dockercfg-mtjrs\"" Jul 05 13:53:14.030879 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.030846 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh"] Jul 05 13:53:14.096066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.096026 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-workload-certs\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.096066 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.096068 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/771aeb83-287c-448f-afae-866419ed11e2-istio-podinfo\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.096301 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.096134 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvc4c\" (UniqueName: \"kubernetes.io/projected/771aeb83-287c-448f-afae-866419ed11e2-kube-api-access-jvc4c\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.096301 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.096180 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-istio-data\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.096301 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.096216 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/771aeb83-287c-448f-afae-866419ed11e2-istiod-ca-cert\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.096301 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.096251 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-credential-socket\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.096301 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.096300 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/771aeb83-287c-448f-afae-866419ed11e2-istio-token\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.096530 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.096335 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-workload-socket\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.096530 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.096357 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-istio-envoy\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.198115 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.198075 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-workload-certs\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.198340 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.198137 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/771aeb83-287c-448f-afae-866419ed11e2-istio-podinfo\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.198340 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.198185 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvc4c\" (UniqueName: \"kubernetes.io/projected/771aeb83-287c-448f-afae-866419ed11e2-kube-api-access-jvc4c\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.198340 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.198229 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-istio-data\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.198340 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.198271 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/771aeb83-287c-448f-afae-866419ed11e2-istiod-ca-cert\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.198340 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.198299 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-credential-socket\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.198340 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.198341 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/771aeb83-287c-448f-afae-866419ed11e2-istio-token\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.198706 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.198379 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-workload-socket\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.198706 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.198401 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-istio-envoy\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.198706 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.198563 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-workload-certs\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.199808 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.198913 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-credential-socket\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.199808 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.199174 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-workload-socket\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.199808 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.199452 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/771aeb83-287c-448f-afae-866419ed11e2-istiod-ca-cert\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.199808 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.199765 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-istio-data\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.202537 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.202510 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/771aeb83-287c-448f-afae-866419ed11e2-istio-podinfo\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.202694 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.202564 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-istio-envoy\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.208952 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.208915 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/771aeb83-287c-448f-afae-866419ed11e2-istio-token\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.209103 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.209027 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvc4c\" (UniqueName: \"kubernetes.io/projected/771aeb83-287c-448f-afae-866419ed11e2-kube-api-access-jvc4c\") pod \"e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.325075 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.324976 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:14.491226 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.491196 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh"] Jul 05 13:53:14.493408 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:53:14.493363 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod771aeb83_287c_448f_afae_866419ed11e2.slice/crio-26061b5dd2f3100cb7393d52029e9acfa6c3b5d5a3f93e8e372705b99aee2ea1 WatchSource:0}: Error finding container 26061b5dd2f3100cb7393d52029e9acfa6c3b5d5a3f93e8e372705b99aee2ea1: Status 404 returned error can't find the container with id 26061b5dd2f3100cb7393d52029e9acfa6c3b5d5a3f93e8e372705b99aee2ea1 Jul 05 13:53:14.496162 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.496121 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:53:14.496257 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.496215 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:53:14.496337 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.496258 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:53:14.918062 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.918026 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" event={"ID":"771aeb83-287c-448f-afae-866419ed11e2","Type":"ContainerStarted","Data":"e65df84eca2e9832443e99fabfd1c8bd76be39a07f3dfa0c3e3d103d59d3babd"} Jul 05 13:53:14.918264 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.918081 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" event={"ID":"771aeb83-287c-448f-afae-866419ed11e2","Type":"ContainerStarted","Data":"26061b5dd2f3100cb7393d52029e9acfa6c3b5d5a3f93e8e372705b99aee2ea1"} Jul 05 13:53:14.946057 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:14.946003 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" podStartSLOduration=1.945986236 podStartE2EDuration="1.945986236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:53:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:53:14.944077389 +0000 UTC m=+2081.285095164" watchObservedRunningTime="2026-07-05 13:53:14.945986236 +0000 UTC m=+2081.287004008" Jul 05 13:53:15.326002 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:15.325896 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:15.331801 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:15.331770 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:15.924034 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:15.924002 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:15.925126 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:15.925106 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:23.906980 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:23.906928 2567 patch_prober.go:28] interesting pod/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 05 13:53:23.907450 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:23.907019 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" podUID="3416d16d-35d4-4b1d-8065-0013e0727b0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 05 13:53:25.987178 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:25.987139 2567 generic.go:356] "Generic (PLEG): container finished" podID="3416d16d-35d4-4b1d-8065-0013e0727b0c" containerID="8f06c2e9feafd22f60ad4cb61bf99d2895e95002b3aa931b358ef540ce188149" exitCode=0 Jul 05 13:53:25.987648 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:25.987209 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" event={"ID":"3416d16d-35d4-4b1d-8065-0013e0727b0c","Type":"ContainerDied","Data":"8f06c2e9feafd22f60ad4cb61bf99d2895e95002b3aa931b358ef540ce188149"} Jul 05 13:53:25.987648 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:25.987636 2567 scope.go:117] "RemoveContainer" containerID="8f06c2e9feafd22f60ad4cb61bf99d2895e95002b3aa931b358ef540ce188149" Jul 05 13:53:25.988862 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:25.988830 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:53:25.989008 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:25.988919 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:53:25.989008 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:25.988969 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:53:27.004195 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:27.004138 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" event={"ID":"3416d16d-35d4-4b1d-8065-0013e0727b0c","Type":"ContainerStarted","Data":"e1cbf2ecfea78a60ae57a7063af9246d233e515c0bea73b6fed1de24dbf8ac98"} Jul 05 13:53:27.327170 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:27.327077 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:27.327323 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:27.327258 2567 patch_prober.go:28] interesting pod/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 05 13:53:27.327366 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:27.327314 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" podUID="3416d16d-35d4-4b1d-8065-0013e0727b0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 05 13:53:28.327702 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:28.327646 2567 patch_prober.go:28] interesting pod/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 05 13:53:28.328250 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:28.327725 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" podUID="3416d16d-35d4-4b1d-8065-0013e0727b0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 05 13:53:29.326979 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:29.326929 2567 patch_prober.go:28] interesting pod/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 05 13:53:29.327298 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:29.327269 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" podUID="3416d16d-35d4-4b1d-8065-0013e0727b0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 05 13:53:30.327013 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:30.326962 2567 patch_prober.go:28] interesting pod/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 05 13:53:30.327552 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:30.327039 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" podUID="3416d16d-35d4-4b1d-8065-0013e0727b0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 05 13:53:31.330301 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:31.330270 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:31.330774 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:31.330678 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:31.331337 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:31.331311 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:53:34.343313 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:34.343275 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-26jcf_bccc5e19-26b8-4f6b-96f2-92684dc3d25a/console-operator/1.log" Jul 05 13:53:34.348617 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:34.348587 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-26jcf_bccc5e19-26b8-4f6b-96f2-92684dc3d25a/console-operator/1.log" Jul 05 13:53:34.350897 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:34.350875 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swckn_a447ce81-b3cb-4269-8d46-e6e612b89e0b/ovn-acl-logging/0.log" Jul 05 13:53:34.355654 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:34.355634 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swckn_a447ce81-b3cb-4269-8d46-e6e612b89e0b/ovn-acl-logging/0.log" Jul 05 13:53:35.064636 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:35.064550 2567 generic.go:356] "Generic (PLEG): container finished" podID="771aeb83-287c-448f-afae-866419ed11e2" containerID="e65df84eca2e9832443e99fabfd1c8bd76be39a07f3dfa0c3e3d103d59d3babd" exitCode=0 Jul 05 13:53:35.064636 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:35.064619 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" event={"ID":"771aeb83-287c-448f-afae-866419ed11e2","Type":"ContainerDied","Data":"e65df84eca2e9832443e99fabfd1c8bd76be39a07f3dfa0c3e3d103d59d3babd"} Jul 05 13:53:35.065089 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:35.065075 2567 scope.go:117] "RemoveContainer" containerID="e65df84eca2e9832443e99fabfd1c8bd76be39a07f3dfa0c3e3d103d59d3babd" Jul 05 13:53:35.066120 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:35.066087 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:53:35.066240 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:35.066178 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:53:35.066240 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:35.066227 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:53:36.072500 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:36.072462 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" event={"ID":"771aeb83-287c-448f-afae-866419ed11e2","Type":"ContainerStarted","Data":"54d4d5dcce2dccd74abf865a4f81a748f72b10d304e7aad96b54d80e96e31dbe"} Jul 05 13:53:36.326124 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:36.326022 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:36.326362 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:36.326293 2567 patch_prober.go:28] interesting pod/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 05 13:53:36.326362 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:36.326350 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" podUID="771aeb83-287c-448f-afae-866419ed11e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 05 13:53:37.326132 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:37.326089 2567 patch_prober.go:28] interesting pod/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 05 13:53:37.326528 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:37.326151 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" podUID="771aeb83-287c-448f-afae-866419ed11e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 05 13:53:38.325422 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:38.325379 2567 patch_prober.go:28] interesting pod/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 05 13:53:38.325620 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:38.325442 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" podUID="771aeb83-287c-448f-afae-866419ed11e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 05 13:53:39.325418 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:39.325380 2567 patch_prober.go:28] interesting pod/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 05 13:53:39.325864 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:39.325441 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" podUID="771aeb83-287c-448f-afae-866419ed11e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 05 13:53:40.329075 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:40.329041 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:40.329557 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:40.329354 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:53:40.330087 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:53:40.330068 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:58:34.382440 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:58:34.382403 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-26jcf_bccc5e19-26b8-4f6b-96f2-92684dc3d25a/console-operator/1.log" Jul 05 13:58:34.389529 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:58:34.389498 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-26jcf_bccc5e19-26b8-4f6b-96f2-92684dc3d25a/console-operator/1.log" Jul 05 13:58:34.390121 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:58:34.390096 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swckn_a447ce81-b3cb-4269-8d46-e6e612b89e0b/ovn-acl-logging/0.log" Jul 05 13:58:34.398296 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:58:34.398270 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swckn_a447ce81-b3cb-4269-8d46-e6e612b89e0b/ovn-acl-logging/0.log" Jul 05 13:59:20.043541 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:20.043495 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq"] Jul 05 13:59:20.044210 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:20.043807 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" podUID="3416d16d-35d4-4b1d-8065-0013e0727b0c" containerName="istio-proxy" containerID="cri-o://e1cbf2ecfea78a60ae57a7063af9246d233e515c0bea73b6fed1de24dbf8ac98" gracePeriod=30 Jul 05 13:59:25.311897 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.311816 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:59:25.396641 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.396601 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-istio-data\") pod \"3416d16d-35d4-4b1d-8065-0013e0727b0c\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " Jul 05 13:59:25.396867 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.396653 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3416d16d-35d4-4b1d-8065-0013e0727b0c-istio-token\") pod \"3416d16d-35d4-4b1d-8065-0013e0727b0c\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " Jul 05 13:59:25.396867 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.396676 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzbww\" (UniqueName: \"kubernetes.io/projected/3416d16d-35d4-4b1d-8065-0013e0727b0c-kube-api-access-wzbww\") pod \"3416d16d-35d4-4b1d-8065-0013e0727b0c\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " Jul 05 13:59:25.396867 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.396794 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-workload-socket\") pod \"3416d16d-35d4-4b1d-8065-0013e0727b0c\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " Jul 05 13:59:25.396867 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.396851 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3416d16d-35d4-4b1d-8065-0013e0727b0c-istio-podinfo\") pod \"3416d16d-35d4-4b1d-8065-0013e0727b0c\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " Jul 05 13:59:25.397084 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.396874 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-istio-envoy\") pod \"3416d16d-35d4-4b1d-8065-0013e0727b0c\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " Jul 05 13:59:25.397084 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.396951 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-workload-certs\") pod \"3416d16d-35d4-4b1d-8065-0013e0727b0c\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " Jul 05 13:59:25.397084 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.397006 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3416d16d-35d4-4b1d-8065-0013e0727b0c-istiod-ca-cert\") pod \"3416d16d-35d4-4b1d-8065-0013e0727b0c\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " Jul 05 13:59:25.397084 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.397057 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-credential-socket\") pod \"3416d16d-35d4-4b1d-8065-0013e0727b0c\" (UID: \"3416d16d-35d4-4b1d-8065-0013e0727b0c\") " Jul 05 13:59:25.397680 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.397604 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3416d16d-35d4-4b1d-8065-0013e0727b0c" (UID: "3416d16d-35d4-4b1d-8065-0013e0727b0c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:59:25.397680 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.397624 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3416d16d-35d4-4b1d-8065-0013e0727b0c" (UID: "3416d16d-35d4-4b1d-8065-0013e0727b0c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:59:25.397680 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.397605 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-istio-data" (OuterVolumeSpecName: "istio-data") pod "3416d16d-35d4-4b1d-8065-0013e0727b0c" (UID: "3416d16d-35d4-4b1d-8065-0013e0727b0c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:59:25.397970 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.397829 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3416d16d-35d4-4b1d-8065-0013e0727b0c" (UID: "3416d16d-35d4-4b1d-8065-0013e0727b0c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:59:25.397970 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.397870 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3416d16d-35d4-4b1d-8065-0013e0727b0c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3416d16d-35d4-4b1d-8065-0013e0727b0c" (UID: "3416d16d-35d4-4b1d-8065-0013e0727b0c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:59:25.399318 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.399294 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3416d16d-35d4-4b1d-8065-0013e0727b0c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3416d16d-35d4-4b1d-8065-0013e0727b0c" (UID: "3416d16d-35d4-4b1d-8065-0013e0727b0c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 05 13:59:25.399614 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.399585 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3416d16d-35d4-4b1d-8065-0013e0727b0c-kube-api-access-wzbww" (OuterVolumeSpecName: "kube-api-access-wzbww") pod "3416d16d-35d4-4b1d-8065-0013e0727b0c" (UID: "3416d16d-35d4-4b1d-8065-0013e0727b0c"). InnerVolumeSpecName "kube-api-access-wzbww". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:59:25.399614 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.399597 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3416d16d-35d4-4b1d-8065-0013e0727b0c-istio-token" (OuterVolumeSpecName: "istio-token") pod "3416d16d-35d4-4b1d-8065-0013e0727b0c" (UID: "3416d16d-35d4-4b1d-8065-0013e0727b0c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:59:25.399723 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.399613 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3416d16d-35d4-4b1d-8065-0013e0727b0c" (UID: "3416d16d-35d4-4b1d-8065-0013e0727b0c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:59:25.498113 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.498079 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3416d16d-35d4-4b1d-8065-0013e0727b0c-istiod-ca-cert\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:59:25.498113 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.498111 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-credential-socket\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:59:25.498113 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.498121 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-istio-data\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:59:25.498386 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.498129 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3416d16d-35d4-4b1d-8065-0013e0727b0c-istio-token\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:59:25.498386 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.498139 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzbww\" (UniqueName: \"kubernetes.io/projected/3416d16d-35d4-4b1d-8065-0013e0727b0c-kube-api-access-wzbww\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:59:25.498386 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.498149 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-workload-socket\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:59:25.498386 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.498158 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3416d16d-35d4-4b1d-8065-0013e0727b0c-istio-podinfo\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:59:25.498386 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.498167 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-istio-envoy\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:59:25.498386 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:25.498175 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3416d16d-35d4-4b1d-8065-0013e0727b0c-workload-certs\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:59:26.125445 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:26.125404 2567 generic.go:356] "Generic (PLEG): container finished" podID="3416d16d-35d4-4b1d-8065-0013e0727b0c" containerID="e1cbf2ecfea78a60ae57a7063af9246d233e515c0bea73b6fed1de24dbf8ac98" exitCode=0 Jul 05 13:59:26.125626 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:26.125457 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" Jul 05 13:59:26.125626 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:26.125474 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" event={"ID":"3416d16d-35d4-4b1d-8065-0013e0727b0c","Type":"ContainerDied","Data":"e1cbf2ecfea78a60ae57a7063af9246d233e515c0bea73b6fed1de24dbf8ac98"} Jul 05 13:59:26.125626 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:26.125530 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq" event={"ID":"3416d16d-35d4-4b1d-8065-0013e0727b0c","Type":"ContainerDied","Data":"f321d850a2b9c2ad899b48a2d4f1eda5b1d5ad0c19687871bf7d3a637a653d73"} Jul 05 13:59:26.125626 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:26.125562 2567 scope.go:117] "RemoveContainer" containerID="e1cbf2ecfea78a60ae57a7063af9246d233e515c0bea73b6fed1de24dbf8ac98" Jul 05 13:59:26.136558 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:26.136537 2567 scope.go:117] "RemoveContainer" containerID="8f06c2e9feafd22f60ad4cb61bf99d2895e95002b3aa931b358ef540ce188149" Jul 05 13:59:26.148829 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:26.148806 2567 scope.go:117] "RemoveContainer" containerID="e1cbf2ecfea78a60ae57a7063af9246d233e515c0bea73b6fed1de24dbf8ac98" Jul 05 13:59:26.149200 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:59:26.149178 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1cbf2ecfea78a60ae57a7063af9246d233e515c0bea73b6fed1de24dbf8ac98\": container with ID starting with e1cbf2ecfea78a60ae57a7063af9246d233e515c0bea73b6fed1de24dbf8ac98 not found: ID does not exist" containerID="e1cbf2ecfea78a60ae57a7063af9246d233e515c0bea73b6fed1de24dbf8ac98" Jul 05 13:59:26.149260 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:26.149212 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1cbf2ecfea78a60ae57a7063af9246d233e515c0bea73b6fed1de24dbf8ac98"} err="failed to get container status \"e1cbf2ecfea78a60ae57a7063af9246d233e515c0bea73b6fed1de24dbf8ac98\": rpc error: code = NotFound desc = could not find container \"e1cbf2ecfea78a60ae57a7063af9246d233e515c0bea73b6fed1de24dbf8ac98\": container with ID starting with e1cbf2ecfea78a60ae57a7063af9246d233e515c0bea73b6fed1de24dbf8ac98 not found: ID does not exist" Jul 05 13:59:26.149260 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:26.149232 2567 scope.go:117] "RemoveContainer" containerID="8f06c2e9feafd22f60ad4cb61bf99d2895e95002b3aa931b358ef540ce188149" Jul 05 13:59:26.149524 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:59:26.149500 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f06c2e9feafd22f60ad4cb61bf99d2895e95002b3aa931b358ef540ce188149\": container with ID starting with 8f06c2e9feafd22f60ad4cb61bf99d2895e95002b3aa931b358ef540ce188149 not found: ID does not exist" containerID="8f06c2e9feafd22f60ad4cb61bf99d2895e95002b3aa931b358ef540ce188149" Jul 05 13:59:26.149566 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:26.149528 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f06c2e9feafd22f60ad4cb61bf99d2895e95002b3aa931b358ef540ce188149"} err="failed to get container status \"8f06c2e9feafd22f60ad4cb61bf99d2895e95002b3aa931b358ef540ce188149\": rpc error: code = NotFound desc = could not find container \"8f06c2e9feafd22f60ad4cb61bf99d2895e95002b3aa931b358ef540ce188149\": container with ID starting with 8f06c2e9feafd22f60ad4cb61bf99d2895e95002b3aa931b358ef540ce188149 not found: ID does not exist" Jul 05 13:59:26.165693 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:26.165655 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq"] Jul 05 13:59:26.189981 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:26.189950 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-6096ec-openshift-default-574b968979-xrwdq"] Jul 05 13:59:26.216760 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:26.216704 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3416d16d-35d4-4b1d-8065-0013e0727b0c" path="/var/lib/kubelet/pods/3416d16d-35d4-4b1d-8065-0013e0727b0c/volumes" Jul 05 13:59:28.108198 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:28.108159 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh"] Jul 05 13:59:28.108689 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:28.108419 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" podUID="771aeb83-287c-448f-afae-866419ed11e2" containerName="istio-proxy" containerID="cri-o://54d4d5dcce2dccd74abf865a4f81a748f72b10d304e7aad96b54d80e96e31dbe" gracePeriod=30 Jul 05 13:59:33.343064 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.343033 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-slvsr_626bf575-c542-415d-9d9a-8480539083c5/manager/0.log" Jul 05 13:59:33.374681 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.374612 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:59:33.460861 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.460830 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7nx7z_83909d91-90dd-4e68-9104-a8d2138b50ec/manager/0.log" Jul 05 13:59:33.486755 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.486715 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvc4c\" (UniqueName: \"kubernetes.io/projected/771aeb83-287c-448f-afae-866419ed11e2-kube-api-access-jvc4c\") pod \"771aeb83-287c-448f-afae-866419ed11e2\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " Jul 05 13:59:33.486951 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.486788 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-credential-socket\") pod \"771aeb83-287c-448f-afae-866419ed11e2\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " Jul 05 13:59:33.486951 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.486851 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-workload-certs\") pod \"771aeb83-287c-448f-afae-866419ed11e2\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " Jul 05 13:59:33.486951 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.486869 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-istio-envoy\") pod \"771aeb83-287c-448f-afae-866419ed11e2\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " Jul 05 13:59:33.486951 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.486912 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-workload-socket\") pod \"771aeb83-287c-448f-afae-866419ed11e2\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " Jul 05 13:59:33.486951 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.486944 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-istio-data\") pod \"771aeb83-287c-448f-afae-866419ed11e2\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " Jul 05 13:59:33.487217 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.486988 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/771aeb83-287c-448f-afae-866419ed11e2-istiod-ca-cert\") pod \"771aeb83-287c-448f-afae-866419ed11e2\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " Jul 05 13:59:33.487217 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.487017 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/771aeb83-287c-448f-afae-866419ed11e2-istio-token\") pod \"771aeb83-287c-448f-afae-866419ed11e2\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " Jul 05 13:59:33.487217 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.487055 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/771aeb83-287c-448f-afae-866419ed11e2-istio-podinfo\") pod \"771aeb83-287c-448f-afae-866419ed11e2\" (UID: \"771aeb83-287c-448f-afae-866419ed11e2\") " Jul 05 13:59:33.487217 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.487168 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "771aeb83-287c-448f-afae-866419ed11e2" (UID: "771aeb83-287c-448f-afae-866419ed11e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:59:33.487217 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.487181 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "771aeb83-287c-448f-afae-866419ed11e2" (UID: "771aeb83-287c-448f-afae-866419ed11e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:59:33.487489 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.487469 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-credential-socket\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:59:33.487542 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.487490 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-workload-certs\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:59:33.487542 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.487472 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/771aeb83-287c-448f-afae-866419ed11e2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "771aeb83-287c-448f-afae-866419ed11e2" (UID: "771aeb83-287c-448f-afae-866419ed11e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:59:33.487542 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.487481 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "771aeb83-287c-448f-afae-866419ed11e2" (UID: "771aeb83-287c-448f-afae-866419ed11e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:59:33.488182 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.488151 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-istio-data" (OuterVolumeSpecName: "istio-data") pod "771aeb83-287c-448f-afae-866419ed11e2" (UID: "771aeb83-287c-448f-afae-866419ed11e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:59:33.489577 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.489549 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/771aeb83-287c-448f-afae-866419ed11e2-kube-api-access-jvc4c" (OuterVolumeSpecName: "kube-api-access-jvc4c") pod "771aeb83-287c-448f-afae-866419ed11e2" (UID: "771aeb83-287c-448f-afae-866419ed11e2"). InnerVolumeSpecName "kube-api-access-jvc4c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:59:33.490112 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.490088 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/771aeb83-287c-448f-afae-866419ed11e2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "771aeb83-287c-448f-afae-866419ed11e2" (UID: "771aeb83-287c-448f-afae-866419ed11e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 05 13:59:33.490203 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.490113 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/771aeb83-287c-448f-afae-866419ed11e2-istio-token" (OuterVolumeSpecName: "istio-token") pod "771aeb83-287c-448f-afae-866419ed11e2" (UID: "771aeb83-287c-448f-afae-866419ed11e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:59:33.490203 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.490085 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "771aeb83-287c-448f-afae-866419ed11e2" (UID: "771aeb83-287c-448f-afae-866419ed11e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:59:33.589051 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.589009 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-istio-envoy\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:59:33.589051 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.589047 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-workload-socket\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:59:33.589051 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.589060 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/771aeb83-287c-448f-afae-866419ed11e2-istio-data\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:59:33.589320 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.589068 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/771aeb83-287c-448f-afae-866419ed11e2-istiod-ca-cert\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:59:33.589320 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.589099 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/771aeb83-287c-448f-afae-866419ed11e2-istio-token\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:59:33.589320 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.589107 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/771aeb83-287c-448f-afae-866419ed11e2-istio-podinfo\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:59:33.589320 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.589119 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvc4c\" (UniqueName: \"kubernetes.io/projected/771aeb83-287c-448f-afae-866419ed11e2-kube-api-access-jvc4c\") on node \"ip-10-0-141-119.ec2.internal\" DevicePath \"\"" Jul 05 13:59:33.918953 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:33.918918 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6bc76f999c-vr2c9_a37fda52-258c-4b8c-8363-62b7abe1d96f/manager/0.log" Jul 05 13:59:34.030954 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:34.030921 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-c4mkr_912df14c-0f0d-4069-b323-ada776d5fd6f/server/0.log" Jul 05 13:59:34.150197 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:34.150163 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-86bkx_21bf44b7-9381-4cbd-a733-b5d85c4f6bdf/manager/0.log" Jul 05 13:59:34.176026 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:34.175925 2567 generic.go:356] "Generic (PLEG): container finished" podID="771aeb83-287c-448f-afae-866419ed11e2" containerID="54d4d5dcce2dccd74abf865a4f81a748f72b10d304e7aad96b54d80e96e31dbe" exitCode=0 Jul 05 13:59:34.176026 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:34.175976 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" Jul 05 13:59:34.176248 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:34.176018 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" event={"ID":"771aeb83-287c-448f-afae-866419ed11e2","Type":"ContainerDied","Data":"54d4d5dcce2dccd74abf865a4f81a748f72b10d304e7aad96b54d80e96e31dbe"} Jul 05 13:59:34.176248 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:34.176105 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh" event={"ID":"771aeb83-287c-448f-afae-866419ed11e2","Type":"ContainerDied","Data":"26061b5dd2f3100cb7393d52029e9acfa6c3b5d5a3f93e8e372705b99aee2ea1"} Jul 05 13:59:34.176248 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:34.176134 2567 scope.go:117] "RemoveContainer" containerID="54d4d5dcce2dccd74abf865a4f81a748f72b10d304e7aad96b54d80e96e31dbe" Jul 05 13:59:34.187061 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:34.187041 2567 scope.go:117] "RemoveContainer" containerID="e65df84eca2e9832443e99fabfd1c8bd76be39a07f3dfa0c3e3d103d59d3babd" Jul 05 13:59:34.196398 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:34.196331 2567 scope.go:117] "RemoveContainer" containerID="54d4d5dcce2dccd74abf865a4f81a748f72b10d304e7aad96b54d80e96e31dbe" Jul 05 13:59:34.196756 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:59:34.196718 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54d4d5dcce2dccd74abf865a4f81a748f72b10d304e7aad96b54d80e96e31dbe\": container with ID starting with 54d4d5dcce2dccd74abf865a4f81a748f72b10d304e7aad96b54d80e96e31dbe not found: ID does not exist" containerID="54d4d5dcce2dccd74abf865a4f81a748f72b10d304e7aad96b54d80e96e31dbe" Jul 05 13:59:34.196867 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:34.196779 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54d4d5dcce2dccd74abf865a4f81a748f72b10d304e7aad96b54d80e96e31dbe"} err="failed to get container status \"54d4d5dcce2dccd74abf865a4f81a748f72b10d304e7aad96b54d80e96e31dbe\": rpc error: code = NotFound desc = could not find container \"54d4d5dcce2dccd74abf865a4f81a748f72b10d304e7aad96b54d80e96e31dbe\": container with ID starting with 54d4d5dcce2dccd74abf865a4f81a748f72b10d304e7aad96b54d80e96e31dbe not found: ID does not exist" Jul 05 13:59:34.196867 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:34.196802 2567 scope.go:117] "RemoveContainer" containerID="e65df84eca2e9832443e99fabfd1c8bd76be39a07f3dfa0c3e3d103d59d3babd" Jul 05 13:59:34.197106 ip-10-0-141-119 kubenswrapper[2567]: E0705 13:59:34.197063 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e65df84eca2e9832443e99fabfd1c8bd76be39a07f3dfa0c3e3d103d59d3babd\": container with ID starting with e65df84eca2e9832443e99fabfd1c8bd76be39a07f3dfa0c3e3d103d59d3babd not found: ID does not exist" containerID="e65df84eca2e9832443e99fabfd1c8bd76be39a07f3dfa0c3e3d103d59d3babd" Jul 05 13:59:34.197106 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:34.197083 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e65df84eca2e9832443e99fabfd1c8bd76be39a07f3dfa0c3e3d103d59d3babd"} err="failed to get container status \"e65df84eca2e9832443e99fabfd1c8bd76be39a07f3dfa0c3e3d103d59d3babd\": rpc error: code = NotFound desc = could not find container \"e65df84eca2e9832443e99fabfd1c8bd76be39a07f3dfa0c3e3d103d59d3babd\": container with ID starting with e65df84eca2e9832443e99fabfd1c8bd76be39a07f3dfa0c3e3d103d59d3babd not found: ID does not exist" Jul 05 13:59:34.200004 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:34.199980 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh"] Jul 05 13:59:34.203850 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:34.203822 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-6579e1-openshift-default-7f48fc94dc-dl2gh"] Jul 05 13:59:34.216358 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:34.216330 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="771aeb83-287c-448f-afae-866419ed11e2" path="/var/lib/kubelet/pods/771aeb83-287c-448f-afae-866419ed11e2/volumes" Jul 05 13:59:34.381183 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:34.381156 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b4f468d94-hm24j_cc491ec3-6f95-4406-8abd-2c7a3d665eeb/copy-manifests/0.log" Jul 05 13:59:34.388003 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:34.387975 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b4f468d94-hm24j_cc491ec3-6f95-4406-8abd-2c7a3d665eeb/manager/0.log" Jul 05 13:59:35.973873 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:35.973845 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-zpsjv_132eb69a-8cde-4a80-a386-7b101e2285ff/manager/0.log" Jul 05 13:59:36.085892 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:36.085859 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-pzwwt_73e4e360-0c3b-4bdf-82e9-b6ce19235329/manager/0.log" Jul 05 13:59:36.318355 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:36.318262 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-r8cgn_8caa73a3-2ab8-43b5-abb6-3adb2cf9e7e2/registry-server/0.log" Jul 05 13:59:36.429526 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:36.429487 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-ptddt_8e6f3d44-83ad-49a3-812f-dc6751f022d7/manager/0.log" Jul 05 13:59:36.651355 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:36.651271 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-9rcph_36c457af-8be7-4661-8eec-b4d158f81993/manager/0.log" Jul 05 13:59:36.996081 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:36.996052 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4pg8k2_8ae1e965-9222-4287-8542-1ab0d3b923c5/istio-proxy/0.log" Jul 05 13:59:37.215698 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:37.215654 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67cd4f9bbf-9q6vw_4226538c-272c-4a18-a7aa-6da1545c1907/kube-auth-proxy/0.log" Jul 05 13:59:37.441165 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:37.441124 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hpstc_2b9cdd17-b41d-4c8f-977f-7b70a30ddb94/istio-proxy/1.log" Jul 05 13:59:37.555502 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:37.555468 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-lzjlh_bd314b66-0a56-4004-977c-55bd446b6ea7/payload-pre-processing/0.log" Jul 05 13:59:37.666231 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:37.666199 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-4zs7b_73e01b28-e127-4f30-9cc8-f1cab122f548/payload-processing/0.log" Jul 05 13:59:37.779016 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:37.778924 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-644877d48f-jctfg_95b0d0a1-07c4-4235-b8d0-5998a2d26473/router/0.log" Jul 05 13:59:38.226820 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:38.226787 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-ddf54_623e80be-3ca7-4872-9519-d6d70c67a29a/main/0.log" Jul 05 13:59:38.444046 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:38.444016 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqww9w_57e047d9-14a2-40fb-8fbf-2c303ec2af37/main/0.log" Jul 05 13:59:45.139854 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:45.139823 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-k92vt_32a4a45d-3b64-476e-9147-09693f809841/global-pull-secret-syncer/0.log" Jul 05 13:59:45.286205 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:45.286169 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qmf78_df0a5b01-f000-4879-8fcc-175f435bce3e/konnectivity-agent/0.log" Jul 05 13:59:45.386342 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:45.386299 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-119.ec2.internal_a1a32c2f892e1c59cdaebdb361cae096/haproxy/0.log" Jul 05 13:59:50.682571 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:50.682537 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-zpsjv_132eb69a-8cde-4a80-a386-7b101e2285ff/manager/0.log" Jul 05 13:59:50.714834 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:50.714804 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-pzwwt_73e4e360-0c3b-4bdf-82e9-b6ce19235329/manager/0.log" Jul 05 13:59:50.779552 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:50.779518 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-r8cgn_8caa73a3-2ab8-43b5-abb6-3adb2cf9e7e2/registry-server/0.log" Jul 05 13:59:50.886038 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:50.885989 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-ptddt_8e6f3d44-83ad-49a3-812f-dc6751f022d7/manager/0.log" Jul 05 13:59:50.986067 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:50.985982 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-9rcph_36c457af-8be7-4661-8eec-b4d158f81993/manager/0.log" Jul 05 13:59:52.834508 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:52.834432 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_466e5064-d6a9-4a02-b37f-0d3e47a7fa0e/alertmanager/0.log" Jul 05 13:59:52.860069 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:52.860026 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_466e5064-d6a9-4a02-b37f-0d3e47a7fa0e/config-reloader/0.log" Jul 05 13:59:52.884672 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:52.884625 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_466e5064-d6a9-4a02-b37f-0d3e47a7fa0e/kube-rbac-proxy-web/0.log" Jul 05 13:59:52.914022 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:52.913990 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_466e5064-d6a9-4a02-b37f-0d3e47a7fa0e/kube-rbac-proxy/0.log" Jul 05 13:59:52.947003 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:52.946960 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_466e5064-d6a9-4a02-b37f-0d3e47a7fa0e/kube-rbac-proxy-metric/0.log" Jul 05 13:59:52.971469 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:52.971444 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_466e5064-d6a9-4a02-b37f-0d3e47a7fa0e/prom-label-proxy/0.log" Jul 05 13:59:52.995648 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:52.995620 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_466e5064-d6a9-4a02-b37f-0d3e47a7fa0e/init-config-reloader/0.log" Jul 05 13:59:53.035888 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:53.035857 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-f8jvx_352ab893-ca08-40ef-9a02-583440eb4788/cluster-monitoring-operator/0.log" Jul 05 13:59:53.508516 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:53.508439 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mkz5g_6b7a4d23-a0e2-45c8-969e-ef29063451b8/node-exporter/0.log" Jul 05 13:59:53.546135 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:53.546107 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mkz5g_6b7a4d23-a0e2-45c8-969e-ef29063451b8/kube-rbac-proxy/0.log" Jul 05 13:59:53.568314 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:53.568278 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mkz5g_6b7a4d23-a0e2-45c8-969e-ef29063451b8/init-textfile/0.log" Jul 05 13:59:53.683707 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:53.683671 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cd65a853-f61d-481b-ba5f-7ac4d667beb4/prometheus/0.log" Jul 05 13:59:53.713053 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:53.713019 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cd65a853-f61d-481b-ba5f-7ac4d667beb4/config-reloader/0.log" Jul 05 13:59:53.742783 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:53.742721 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cd65a853-f61d-481b-ba5f-7ac4d667beb4/thanos-sidecar/0.log" Jul 05 13:59:53.762350 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:53.762249 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cd65a853-f61d-481b-ba5f-7ac4d667beb4/kube-rbac-proxy-web/0.log" Jul 05 13:59:53.784353 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:53.784326 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cd65a853-f61d-481b-ba5f-7ac4d667beb4/kube-rbac-proxy/0.log" Jul 05 13:59:53.807989 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:53.807955 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cd65a853-f61d-481b-ba5f-7ac4d667beb4/kube-rbac-proxy-thanos/0.log" Jul 05 13:59:53.830186 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:53.830159 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cd65a853-f61d-481b-ba5f-7ac4d667beb4/init-config-reloader/0.log" Jul 05 13:59:53.864348 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:53.864320 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-97hz7_e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d/prometheus-operator/0.log" Jul 05 13:59:53.891009 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:53.890981 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-97hz7_e9fa9507-2afd-44c3-980a-6d3b9fbbfd5d/kube-rbac-proxy/0.log" Jul 05 13:59:54.037278 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:54.037178 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78d9c755bc-rnqlg_e998c650-79fc-453d-8f79-71ae0703303e/thanos-query/0.log" Jul 05 13:59:54.059319 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:54.059282 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78d9c755bc-rnqlg_e998c650-79fc-453d-8f79-71ae0703303e/kube-rbac-proxy-web/0.log" Jul 05 13:59:54.079902 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:54.079877 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78d9c755bc-rnqlg_e998c650-79fc-453d-8f79-71ae0703303e/kube-rbac-proxy/0.log" Jul 05 13:59:54.098893 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:54.098858 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78d9c755bc-rnqlg_e998c650-79fc-453d-8f79-71ae0703303e/prom-label-proxy/0.log" Jul 05 13:59:54.122968 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:54.122942 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78d9c755bc-rnqlg_e998c650-79fc-453d-8f79-71ae0703303e/kube-rbac-proxy-rules/0.log" Jul 05 13:59:54.143386 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:54.143356 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78d9c755bc-rnqlg_e998c650-79fc-453d-8f79-71ae0703303e/kube-rbac-proxy-metrics/0.log" Jul 05 13:59:55.056085 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.056050 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tjmkf/perf-node-gather-daemonset-dsbcc"] Jul 05 13:59:55.056533 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.056518 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3416d16d-35d4-4b1d-8065-0013e0727b0c" containerName="istio-proxy" Jul 05 13:59:55.056533 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.056534 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3416d16d-35d4-4b1d-8065-0013e0727b0c" containerName="istio-proxy" Jul 05 13:59:55.056608 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.056542 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="771aeb83-287c-448f-afae-866419ed11e2" containerName="istio-proxy" Jul 05 13:59:55.056608 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.056548 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="771aeb83-287c-448f-afae-866419ed11e2" containerName="istio-proxy" Jul 05 13:59:55.056608 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.056555 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3416d16d-35d4-4b1d-8065-0013e0727b0c" containerName="istio-proxy" Jul 05 13:59:55.056608 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.056561 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3416d16d-35d4-4b1d-8065-0013e0727b0c" containerName="istio-proxy" Jul 05 13:59:55.056608 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.056581 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="771aeb83-287c-448f-afae-866419ed11e2" containerName="istio-proxy" Jul 05 13:59:55.056608 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.056586 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="771aeb83-287c-448f-afae-866419ed11e2" containerName="istio-proxy" Jul 05 13:59:55.056809 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.056656 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3416d16d-35d4-4b1d-8065-0013e0727b0c" containerName="istio-proxy" Jul 05 13:59:55.056809 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.056663 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="771aeb83-287c-448f-afae-866419ed11e2" containerName="istio-proxy" Jul 05 13:59:55.056809 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.056669 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="771aeb83-287c-448f-afae-866419ed11e2" containerName="istio-proxy" Jul 05 13:59:55.056809 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.056675 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3416d16d-35d4-4b1d-8065-0013e0727b0c" containerName="istio-proxy" Jul 05 13:59:55.060172 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.060149 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-dsbcc" Jul 05 13:59:55.063228 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.063206 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tjmkf\"/\"openshift-service-ca.crt\"" Jul 05 13:59:55.063332 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.063302 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tjmkf\"/\"kube-root-ca.crt\"" Jul 05 13:59:55.064350 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.064332 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tjmkf\"/\"default-dockercfg-6mf8z\"" Jul 05 13:59:55.075119 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.075089 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tjmkf/perf-node-gather-daemonset-dsbcc"] Jul 05 13:59:55.104718 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.104677 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da412196-c65d-4311-9c6c-8e888cc1d7d2-lib-modules\") pod \"perf-node-gather-daemonset-dsbcc\" (UID: \"da412196-c65d-4311-9c6c-8e888cc1d7d2\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-dsbcc" Jul 05 13:59:55.104718 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.104723 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/da412196-c65d-4311-9c6c-8e888cc1d7d2-podres\") pod \"perf-node-gather-daemonset-dsbcc\" (UID: \"da412196-c65d-4311-9c6c-8e888cc1d7d2\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-dsbcc" Jul 05 13:59:55.104970 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.104804 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da412196-c65d-4311-9c6c-8e888cc1d7d2-sys\") pod \"perf-node-gather-daemonset-dsbcc\" (UID: \"da412196-c65d-4311-9c6c-8e888cc1d7d2\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-dsbcc" Jul 05 13:59:55.104970 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.104829 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/da412196-c65d-4311-9c6c-8e888cc1d7d2-proc\") pod \"perf-node-gather-daemonset-dsbcc\" (UID: \"da412196-c65d-4311-9c6c-8e888cc1d7d2\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-dsbcc" Jul 05 13:59:55.104970 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.104854 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wwwz\" (UniqueName: \"kubernetes.io/projected/da412196-c65d-4311-9c6c-8e888cc1d7d2-kube-api-access-2wwwz\") pod \"perf-node-gather-daemonset-dsbcc\" (UID: \"da412196-c65d-4311-9c6c-8e888cc1d7d2\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-dsbcc" Jul 05 13:59:55.205415 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.205375 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da412196-c65d-4311-9c6c-8e888cc1d7d2-sys\") pod \"perf-node-gather-daemonset-dsbcc\" (UID: \"da412196-c65d-4311-9c6c-8e888cc1d7d2\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-dsbcc" Jul 05 13:59:55.205415 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.205425 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/da412196-c65d-4311-9c6c-8e888cc1d7d2-proc\") pod \"perf-node-gather-daemonset-dsbcc\" (UID: \"da412196-c65d-4311-9c6c-8e888cc1d7d2\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-dsbcc" Jul 05 13:59:55.205668 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.205460 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wwwz\" (UniqueName: \"kubernetes.io/projected/da412196-c65d-4311-9c6c-8e888cc1d7d2-kube-api-access-2wwwz\") pod \"perf-node-gather-daemonset-dsbcc\" (UID: \"da412196-c65d-4311-9c6c-8e888cc1d7d2\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-dsbcc" Jul 05 13:59:55.205668 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.205499 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da412196-c65d-4311-9c6c-8e888cc1d7d2-lib-modules\") pod \"perf-node-gather-daemonset-dsbcc\" (UID: \"da412196-c65d-4311-9c6c-8e888cc1d7d2\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-dsbcc" Jul 05 13:59:55.205668 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.205526 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da412196-c65d-4311-9c6c-8e888cc1d7d2-sys\") pod \"perf-node-gather-daemonset-dsbcc\" (UID: \"da412196-c65d-4311-9c6c-8e888cc1d7d2\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-dsbcc" Jul 05 13:59:55.205668 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.205528 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/da412196-c65d-4311-9c6c-8e888cc1d7d2-podres\") pod \"perf-node-gather-daemonset-dsbcc\" (UID: \"da412196-c65d-4311-9c6c-8e888cc1d7d2\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-dsbcc" Jul 05 13:59:55.205668 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.205583 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/da412196-c65d-4311-9c6c-8e888cc1d7d2-proc\") pod \"perf-node-gather-daemonset-dsbcc\" (UID: \"da412196-c65d-4311-9c6c-8e888cc1d7d2\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-dsbcc" Jul 05 13:59:55.205668 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.205627 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/da412196-c65d-4311-9c6c-8e888cc1d7d2-podres\") pod \"perf-node-gather-daemonset-dsbcc\" (UID: \"da412196-c65d-4311-9c6c-8e888cc1d7d2\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-dsbcc" Jul 05 13:59:55.205668 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.205653 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da412196-c65d-4311-9c6c-8e888cc1d7d2-lib-modules\") pod \"perf-node-gather-daemonset-dsbcc\" (UID: \"da412196-c65d-4311-9c6c-8e888cc1d7d2\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-dsbcc" Jul 05 13:59:55.214846 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.214825 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wwwz\" (UniqueName: \"kubernetes.io/projected/da412196-c65d-4311-9c6c-8e888cc1d7d2-kube-api-access-2wwwz\") pod \"perf-node-gather-daemonset-dsbcc\" (UID: \"da412196-c65d-4311-9c6c-8e888cc1d7d2\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-dsbcc" Jul 05 13:59:55.302300 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.302272 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-xf8bg_4af1596e-0aa4-4395-839e-49cf9f20ab11/networking-console-plugin/0.log" Jul 05 13:59:55.370930 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.370839 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-dsbcc" Jul 05 13:59:55.522225 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.522197 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tjmkf/perf-node-gather-daemonset-dsbcc"] Jul 05 13:59:55.524049 ip-10-0-141-119 kubenswrapper[2567]: W0705 13:59:55.524019 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podda412196_c65d_4311_9c6c_8e888cc1d7d2.slice/crio-fcbfbb28e4fabe44745e6eee4b4b58c7f27ff9f4593847a8616a9930cbf96feb WatchSource:0}: Error finding container fcbfbb28e4fabe44745e6eee4b4b58c7f27ff9f4593847a8616a9930cbf96feb: Status 404 returned error can't find the container with id fcbfbb28e4fabe44745e6eee4b4b58c7f27ff9f4593847a8616a9930cbf96feb Jul 05 13:59:55.883865 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.883836 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-26jcf_bccc5e19-26b8-4f6b-96f2-92684dc3d25a/console-operator/1.log" Jul 05 13:59:55.889575 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:55.889544 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-26jcf_bccc5e19-26b8-4f6b-96f2-92684dc3d25a/console-operator/2.log" Jul 05 13:59:56.296278 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:56.296230 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-dsbcc" event={"ID":"da412196-c65d-4311-9c6c-8e888cc1d7d2","Type":"ContainerStarted","Data":"499fd4f8140be20110ed585fd18c6a5e50b83a96986726d45bb1528b872fa886"} Jul 05 13:59:56.296685 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:56.296293 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-dsbcc" event={"ID":"da412196-c65d-4311-9c6c-8e888cc1d7d2","Type":"ContainerStarted","Data":"fcbfbb28e4fabe44745e6eee4b4b58c7f27ff9f4593847a8616a9930cbf96feb"} Jul 05 13:59:56.296685 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:56.296332 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-dsbcc" Jul 05 13:59:56.314510 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:56.314427 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-dsbcc" podStartSLOduration=1.314410603 podStartE2EDuration="1.314410603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:59:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:59:56.313230077 +0000 UTC m=+2482.654247846" watchObservedRunningTime="2026-07-05 13:59:56.314410603 +0000 UTC m=+2482.655428375" Jul 05 13:59:56.974681 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:56.974648 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-8dj7d_fab50c14-6777-44c0-a7b7-e3b52ea810ca/volume-data-source-validator/0.log" Jul 05 13:59:58.084216 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:58.084156 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mkxp5_4f14e6cf-c089-437b-90b6-9cae0be31c56/dns/0.log" Jul 05 13:59:58.104048 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:58.104020 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mkxp5_4f14e6cf-c089-437b-90b6-9cae0be31c56/kube-rbac-proxy/0.log" Jul 05 13:59:58.132313 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:58.132278 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-78f24_8af4784e-62fd-4a92-ba60-c91f69a2322b/dns-node-resolver/0.log" Jul 05 13:59:58.756480 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:58.756430 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8s4z9_84b36d5b-eb96-4252-a0d2-a1f5d8c58a4e/node-ca/0.log" Jul 05 13:59:59.743314 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:59.743286 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4pg8k2_8ae1e965-9222-4287-8542-1ab0d3b923c5/istio-proxy/0.log" Jul 05 13:59:59.869652 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:59.869622 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67cd4f9bbf-9q6vw_4226538c-272c-4a18-a7aa-6da1545c1907/kube-auth-proxy/0.log" Jul 05 13:59:59.929568 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:59.929533 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hpstc_2b9cdd17-b41d-4c8f-977f-7b70a30ddb94/istio-proxy/0.log" Jul 05 13:59:59.959329 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:59.959282 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hpstc_2b9cdd17-b41d-4c8f-977f-7b70a30ddb94/istio-proxy/1.log" Jul 05 13:59:59.987565 ip-10-0-141-119 kubenswrapper[2567]: I0705 13:59:59.987520 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-lzjlh_bd314b66-0a56-4004-977c-55bd446b6ea7/payload-pre-processing/0.log" Jul 05 14:00:00.034432 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:00.034339 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-4zs7b_73e01b28-e127-4f30-9cc8-f1cab122f548/payload-processing/0.log" Jul 05 14:00:00.070723 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:00.070695 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-644877d48f-jctfg_95b0d0a1-07c4-4235-b8d0-5998a2d26473/router/0.log" Jul 05 14:00:00.823054 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:00.823024 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8k5ph_99c92f04-b25b-45b6-aa87-f7085cc847eb/serve-healthcheck-canary/0.log" Jul 05 14:00:01.339026 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:01.338989 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-7vrt7_532a0da4-c9ae-4e33-96e8-381c6715d7e9/insights-operator/0.log" Jul 05 14:00:01.339535 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:01.339509 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-7vrt7_532a0da4-c9ae-4e33-96e8-381c6715d7e9/insights-operator/1.log" Jul 05 14:00:01.539722 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:01.539684 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ptp7g_524c4576-f63f-40a5-9108-ddc53661f8e8/kube-rbac-proxy/0.log" Jul 05 14:00:01.562046 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:01.562013 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ptp7g_524c4576-f63f-40a5-9108-ddc53661f8e8/exporter/0.log" Jul 05 14:00:01.585281 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:01.585245 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ptp7g_524c4576-f63f-40a5-9108-ddc53661f8e8/extractor/0.log" Jul 05 14:00:02.312232 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:02.312203 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-dsbcc" Jul 05 14:00:04.000825 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:04.000783 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-slvsr_626bf575-c542-415d-9d9a-8480539083c5/manager/0.log" Jul 05 14:00:04.130370 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:04.130335 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7nx7z_83909d91-90dd-4e68-9104-a8d2138b50ec/manager/0.log" Jul 05 14:00:04.353253 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:04.353219 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6bc76f999c-vr2c9_a37fda52-258c-4b8c-8363-62b7abe1d96f/manager/0.log" Jul 05 14:00:04.374982 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:04.374953 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-c4mkr_912df14c-0f0d-4069-b323-ada776d5fd6f/server/0.log" Jul 05 14:00:04.446320 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:04.446284 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-86bkx_21bf44b7-9381-4cbd-a733-b5d85c4f6bdf/manager/0.log" Jul 05 14:00:04.553333 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:04.553301 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b4f468d94-hm24j_cc491ec3-6f95-4406-8abd-2c7a3d665eeb/manager/0.log" Jul 05 14:00:04.578465 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:04.578440 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b4f468d94-hm24j_cc491ec3-6f95-4406-8abd-2c7a3d665eeb/copy-manifests/0.log" Jul 05 14:00:06.044663 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:06.044631 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4pg8k2_8ae1e965-9222-4287-8542-1ab0d3b923c5/istio-proxy/0.log" Jul 05 14:00:06.151515 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:06.151478 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67cd4f9bbf-9q6vw_4226538c-272c-4a18-a7aa-6da1545c1907/kube-auth-proxy/0.log" Jul 05 14:00:06.172780 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:06.172731 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-2wbsv_6852ebdd-87cb-48bc-8b94-da3c61fb1b02/openshift-lws-operator/0.log" Jul 05 14:00:06.189538 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:06.189505 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hpstc_2b9cdd17-b41d-4c8f-977f-7b70a30ddb94/istio-proxy/0.log" Jul 05 14:00:06.220669 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:06.220638 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hpstc_2b9cdd17-b41d-4c8f-977f-7b70a30ddb94/istio-proxy/1.log" Jul 05 14:00:06.244157 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:06.244110 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-lzjlh_bd314b66-0a56-4004-977c-55bd446b6ea7/payload-pre-processing/0.log" Jul 05 14:00:06.281225 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:06.281185 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-4zs7b_73e01b28-e127-4f30-9cc8-f1cab122f548/payload-processing/0.log" Jul 05 14:00:06.297770 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:06.297661 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-644877d48f-jctfg_95b0d0a1-07c4-4235-b8d0-5998a2d26473/router/0.log" Jul 05 14:00:07.443579 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:07.443537 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4pg8k2_8ae1e965-9222-4287-8542-1ab0d3b923c5/istio-proxy/0.log" Jul 05 14:00:07.551495 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:07.551464 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67cd4f9bbf-9q6vw_4226538c-272c-4a18-a7aa-6da1545c1907/kube-auth-proxy/0.log" Jul 05 14:00:07.590118 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:07.590077 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hpstc_2b9cdd17-b41d-4c8f-977f-7b70a30ddb94/istio-proxy/0.log" Jul 05 14:00:07.620806 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:07.620770 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hpstc_2b9cdd17-b41d-4c8f-977f-7b70a30ddb94/istio-proxy/1.log" Jul 05 14:00:07.640489 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:07.640454 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-lzjlh_bd314b66-0a56-4004-977c-55bd446b6ea7/payload-pre-processing/0.log" Jul 05 14:00:07.677299 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:07.677256 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-4zs7b_73e01b28-e127-4f30-9cc8-f1cab122f548/payload-processing/0.log" Jul 05 14:00:07.693358 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:07.693329 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-644877d48f-jctfg_95b0d0a1-07c4-4235-b8d0-5998a2d26473/router/0.log" Jul 05 14:00:11.539918 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:11.539835 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-rgmdx_e21b2ebc-6b56-45d0-ab65-e144505dd4b4/kube-storage-version-migrator-operator/1.log" Jul 05 14:00:11.540843 ip-10-0-141-119 kubenswrapper[2567]: I0705 14:00:11.540820 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-rgmdx_e21b2ebc-6b56-45d0-ab65-e144505dd4b4/kube-storage-version-migrator-operator/0.log"