Jul 09 04:06:49.659598 ip-10-0-130-153 systemd[1]: Starting Kubernetes Kubelet... Jul 09 04:06:50.051408 ip-10-0-130-153 kubenswrapper[2534]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 04:06:50.051408 ip-10-0-130-153 kubenswrapper[2534]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 04:06:50.051408 ip-10-0-130-153 kubenswrapper[2534]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 04:06:50.051408 ip-10-0-130-153 kubenswrapper[2534]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 04:06:50.052070 ip-10-0-130-153 kubenswrapper[2534]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 04:06:50.052420 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.052270 2534 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 04:06:50.056351 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056336 2534 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 04:06:50.056351 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056351 2534 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 04:06:50.056415 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056357 2534 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 04:06:50.056415 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056360 2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 04:06:50.056415 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056363 2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 04:06:50.056415 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056366 2534 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 04:06:50.056415 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056369 2534 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 04:06:50.056415 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056372 2534 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 04:06:50.056415 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056374 2534 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 04:06:50.056415 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056377 2534 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 04:06:50.056415 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056389 2534 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 04:06:50.056415 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056393 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 04:06:50.056415 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056397 2534 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 04:06:50.056415 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056400 2534 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 04:06:50.056415 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056403 2534 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 04:06:50.056415 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056405 2534 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 04:06:50.056415 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056408 2534 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 04:06:50.056415 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056411 2534 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 04:06:50.056415 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056413 2534 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 04:06:50.056415 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056416 2534 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 04:06:50.056415 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056418 2534 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 04:06:50.057095 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056421 2534 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 04:06:50.057095 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056424 2534 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 04:06:50.057095 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056427 2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 04:06:50.057095 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056430 2534 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 04:06:50.057095 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056432 2534 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 04:06:50.057095 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056435 2534 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 04:06:50.057095 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056437 2534 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 04:06:50.057095 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056439 2534 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 04:06:50.057095 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056442 2534 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 04:06:50.057095 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056444 2534 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 04:06:50.057095 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056447 2534 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 04:06:50.057095 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056449 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 04:06:50.057095 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056451 2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 04:06:50.057095 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056456 2534 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 04:06:50.057095 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056459 2534 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 04:06:50.057095 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056462 2534 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 04:06:50.057095 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056465 2534 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 04:06:50.057095 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056468 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 04:06:50.057095 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056470 2534 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 04:06:50.057095 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056473 2534 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 04:06:50.057681 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056476 2534 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 04:06:50.057681 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056478 2534 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 04:06:50.057681 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056515 2534 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 04:06:50.057681 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056548 2534 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 04:06:50.057681 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056551 2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 04:06:50.057681 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056554 2534 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 04:06:50.057681 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.056557 2534 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 04:06:50.057681 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057645 2534 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 04:06:50.057681 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057665 2534 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 04:06:50.057681 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057673 2534 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 04:06:50.057928 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057789 2534 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 04:06:50.057928 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057794 2534 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 04:06:50.057928 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057798 2534 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 04:06:50.057928 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057801 2534 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 04:06:50.057928 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057803 2534 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 04:06:50.057928 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057808 2534 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 04:06:50.057928 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057810 2534 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 04:06:50.057928 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057813 2534 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 04:06:50.057928 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057816 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 04:06:50.057928 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057819 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 04:06:50.057928 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057822 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 04:06:50.057928 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057824 2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 04:06:50.057928 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057827 2534 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 04:06:50.057928 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057830 2534 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 04:06:50.057928 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057833 2534 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 04:06:50.057928 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057835 2534 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 04:06:50.057928 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057838 2534 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 04:06:50.057928 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057840 2534 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 04:06:50.057928 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057843 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 04:06:50.057928 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057845 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 04:06:50.058422 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057848 2534 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 04:06:50.058422 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057850 2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 04:06:50.058422 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057852 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 04:06:50.058422 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057855 2534 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 04:06:50.058422 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057857 2534 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 04:06:50.058422 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057860 2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 04:06:50.058422 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057862 2534 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 04:06:50.058422 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057864 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 04:06:50.058422 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057877 2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 04:06:50.058422 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057879 2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 04:06:50.058422 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057883 2534 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 04:06:50.058422 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057885 2534 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 04:06:50.058422 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057888 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 04:06:50.058422 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057891 2534 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 04:06:50.058422 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057894 2534 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 04:06:50.058422 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057897 2534 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 04:06:50.058422 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057900 2534 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 04:06:50.058422 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057903 2534 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 04:06:50.058422 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057905 2534 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 04:06:50.058422 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057908 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 04:06:50.058895 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057910 2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 04:06:50.058895 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057913 2534 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 04:06:50.058895 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057916 2534 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 04:06:50.058895 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057918 2534 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 04:06:50.058895 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057920 2534 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 04:06:50.058895 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057923 2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 04:06:50.058895 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057925 2534 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 04:06:50.058895 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.057928 2534 feature_gate.go:328] unrecognized feature gate: Example Jul 09 04:06:50.058895 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058646 2534 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 04:06:50.058895 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058653 2534 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 04:06:50.058895 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058656 2534 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 04:06:50.058895 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058659 2534 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 04:06:50.058895 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058662 2534 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 04:06:50.058895 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058665 2534 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 04:06:50.058895 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058668 2534 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 04:06:50.058895 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058670 2534 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 04:06:50.058895 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058673 2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 04:06:50.058895 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058676 2534 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 04:06:50.058895 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058678 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 04:06:50.058895 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058681 2534 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 04:06:50.059391 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058683 2534 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 04:06:50.059391 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058686 2534 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 04:06:50.059391 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058689 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 04:06:50.059391 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058692 2534 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 04:06:50.059391 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058694 2534 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 04:06:50.059391 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058697 2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 04:06:50.059391 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058699 2534 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 04:06:50.059391 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058702 2534 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 04:06:50.059391 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058704 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 04:06:50.059391 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058707 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 04:06:50.059391 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058709 2534 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 04:06:50.059391 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058712 2534 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 04:06:50.059391 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058714 2534 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 04:06:50.059391 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058718 2534 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 04:06:50.059391 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058721 2534 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 04:06:50.059391 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058723 2534 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 04:06:50.059391 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058726 2534 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 04:06:50.059391 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058728 2534 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 04:06:50.059391 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058732 2534 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 04:06:50.059898 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058737 2534 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 04:06:50.059898 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058740 2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 04:06:50.059898 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058743 2534 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 04:06:50.059898 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058745 2534 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 04:06:50.059898 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058748 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 04:06:50.059898 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058751 2534 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 04:06:50.059898 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058778 2534 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 04:06:50.059898 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058782 2534 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 04:06:50.059898 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058786 2534 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 04:06:50.059898 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058789 2534 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 04:06:50.059898 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058791 2534 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 04:06:50.059898 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058794 2534 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 04:06:50.059898 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058798 2534 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 04:06:50.059898 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058802 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 04:06:50.059898 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058806 2534 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 04:06:50.059898 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058808 2534 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 04:06:50.059898 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058811 2534 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 04:06:50.059898 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058814 2534 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 04:06:50.059898 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058816 2534 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 04:06:50.060380 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058819 2534 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 04:06:50.060380 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058821 2534 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 04:06:50.060380 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058824 2534 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 04:06:50.060380 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058826 2534 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 04:06:50.060380 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058828 2534 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 04:06:50.060380 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058831 2534 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 04:06:50.060380 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058836 2534 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 04:06:50.060380 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058840 2534 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 04:06:50.060380 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058843 2534 feature_gate.go:328] unrecognized feature gate: Example Jul 09 04:06:50.060380 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058845 2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 04:06:50.060380 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058849 2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 04:06:50.060380 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058852 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 04:06:50.060380 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058855 2534 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 04:06:50.060380 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058857 2534 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 04:06:50.060380 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058860 2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 04:06:50.060380 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058862 2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 04:06:50.060380 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058865 2534 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 04:06:50.060380 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058867 2534 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 04:06:50.060380 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058870 2534 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 04:06:50.060841 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058872 2534 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 04:06:50.060841 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058875 2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 04:06:50.060841 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058878 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 04:06:50.060841 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058880 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 04:06:50.060841 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058883 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 04:06:50.060841 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058885 2534 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 04:06:50.060841 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058888 2534 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 04:06:50.060841 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058890 2534 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 04:06:50.060841 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058894 2534 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 04:06:50.060841 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058897 2534 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 04:06:50.060841 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058899 2534 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 04:06:50.060841 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058902 2534 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 04:06:50.060841 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058904 2534 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 04:06:50.060841 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058907 2534 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 04:06:50.060841 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058909 2534 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 04:06:50.060841 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058912 2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 04:06:50.060841 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058914 2534 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 04:06:50.060841 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058917 2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 04:06:50.060841 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058920 2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 04:06:50.060841 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058923 2534 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 04:06:50.061437 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058925 2534 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 04:06:50.061437 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058928 2534 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 04:06:50.061437 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058931 2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 04:06:50.061437 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058933 2534 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 04:06:50.061437 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058936 2534 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 04:06:50.061437 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058939 2534 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 04:06:50.061437 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058941 2534 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 04:06:50.061437 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058943 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 04:06:50.061437 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058946 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 04:06:50.061437 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.058948 2534 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 04:06:50.061437 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059337 2534 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 04:06:50.061437 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059342 2534 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 04:06:50.061437 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059345 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 04:06:50.061437 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059347 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 04:06:50.061437 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059350 2534 feature_gate.go:328] unrecognized feature gate: Example Jul 09 04:06:50.061437 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059353 2534 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 04:06:50.061437 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059355 2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 04:06:50.061437 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059358 2534 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 04:06:50.061437 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059360 2534 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 04:06:50.061437 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059363 2534 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 04:06:50.061917 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059365 2534 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 04:06:50.061917 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059368 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 04:06:50.061917 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059370 2534 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 04:06:50.061917 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059373 2534 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 04:06:50.061917 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059375 2534 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 04:06:50.061917 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059377 2534 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 04:06:50.061917 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059380 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 04:06:50.061917 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059382 2534 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 04:06:50.061917 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059385 2534 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 04:06:50.061917 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059387 2534 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 04:06:50.061917 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059390 2534 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 04:06:50.061917 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059393 2534 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 04:06:50.061917 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059397 2534 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 04:06:50.061917 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059400 2534 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 04:06:50.061917 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059403 2534 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 04:06:50.061917 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059406 2534 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 04:06:50.061917 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059409 2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 04:06:50.061917 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059412 2534 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 04:06:50.061917 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059414 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 04:06:50.062388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059417 2534 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 04:06:50.062388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059419 2534 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 04:06:50.062388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059422 2534 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 04:06:50.062388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059424 2534 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 04:06:50.062388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059426 2534 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 04:06:50.062388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059429 2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 04:06:50.062388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059431 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 04:06:50.062388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059434 2534 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 04:06:50.062388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059438 2534 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 04:06:50.062388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059441 2534 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 04:06:50.062388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059444 2534 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 04:06:50.062388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059447 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 04:06:50.062388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059449 2534 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 04:06:50.062388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059452 2534 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 04:06:50.062388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059455 2534 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 04:06:50.062388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059458 2534 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 04:06:50.062388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059460 2534 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 04:06:50.062388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059463 2534 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 04:06:50.062388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059465 2534 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 04:06:50.062845 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059468 2534 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 04:06:50.062845 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059470 2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 04:06:50.062845 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059473 2534 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 04:06:50.062845 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059475 2534 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 04:06:50.062845 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059479 2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 04:06:50.062845 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059481 2534 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 04:06:50.062845 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059484 2534 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 04:06:50.062845 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059486 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 04:06:50.062845 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059489 2534 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 04:06:50.062845 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059493 2534 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 04:06:50.062845 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059495 2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 04:06:50.062845 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059498 2534 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 04:06:50.062845 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059501 2534 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 04:06:50.062845 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059504 2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 04:06:50.062845 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059506 2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 04:06:50.062845 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059509 2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 04:06:50.062845 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059511 2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 04:06:50.062845 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059513 2534 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 04:06:50.062845 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059516 2534 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 04:06:50.062845 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059518 2534 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 04:06:50.063333 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059520 2534 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 04:06:50.063333 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059524 2534 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 04:06:50.063333 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059526 2534 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 04:06:50.063333 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059528 2534 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 04:06:50.063333 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059531 2534 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 04:06:50.063333 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059533 2534 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 04:06:50.063333 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059536 2534 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 04:06:50.063333 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059540 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 04:06:50.063333 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059543 2534 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 04:06:50.063333 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059545 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 04:06:50.063333 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059548 2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 04:06:50.063333 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059550 2534 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 04:06:50.063333 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059552 2534 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 04:06:50.063333 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059555 2534 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 04:06:50.063333 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059557 2534 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 04:06:50.063333 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059560 2534 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 04:06:50.063333 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059564 2534 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 04:06:50.063333 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059566 2534 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 04:06:50.063333 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059569 2534 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 04:06:50.063333 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059571 2534 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 04:06:50.063804 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059574 2534 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 04:06:50.063804 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059576 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 04:06:50.063804 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059578 2534 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 04:06:50.063804 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059581 2534 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 04:06:50.063804 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059584 2534 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 04:06:50.063804 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059587 2534 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 04:06:50.063804 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059589 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 04:06:50.063804 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059592 2534 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 04:06:50.063804 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059594 2534 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 04:06:50.063804 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059597 2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 04:06:50.063804 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.059599 2534 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 04:06:50.063804 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059666 2534 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 04:06:50.063804 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059673 2534 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 04:06:50.063804 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059678 2534 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 04:06:50.063804 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059682 2534 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 04:06:50.063804 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059686 2534 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 04:06:50.063804 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059689 2534 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 04:06:50.063804 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059693 2534 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 04:06:50.063804 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059697 2534 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 04:06:50.063804 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059702 2534 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 04:06:50.063804 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059705 2534 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 04:06:50.064321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059708 2534 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 04:06:50.064321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059711 2534 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 04:06:50.064321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059714 2534 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 04:06:50.064321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059717 2534 flags.go:64] FLAG: --cgroup-root="" Jul 09 04:06:50.064321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059720 2534 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 04:06:50.064321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059723 2534 flags.go:64] FLAG: --client-ca-file="" Jul 09 04:06:50.064321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059725 2534 flags.go:64] FLAG: --cloud-provider="external" Jul 09 04:06:50.064321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059730 2534 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 04:06:50.064321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059734 2534 flags.go:64] FLAG: --cluster-domain="" Jul 09 04:06:50.064321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059737 2534 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 04:06:50.064321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059740 2534 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 04:06:50.064321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059743 2534 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 04:06:50.064321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059746 2534 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 04:06:50.064321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059749 2534 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 04:06:50.064321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059752 2534 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 04:06:50.064321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059755 2534 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 04:06:50.064321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059759 2534 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 04:06:50.064321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059762 2534 flags.go:64] FLAG: --contention-profiling="false" Jul 09 04:06:50.064321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059764 2534 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 04:06:50.064321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059767 2534 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 04:06:50.064321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059770 2534 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 04:06:50.064321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059773 2534 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 04:06:50.064321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059777 2534 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 04:06:50.064321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059780 2534 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059782 2534 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059785 2534 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059788 2534 flags.go:64] FLAG: --enable-server="true" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059790 2534 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059794 2534 flags.go:64] FLAG: --event-burst="100" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059797 2534 flags.go:64] FLAG: --event-qps="50" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059801 2534 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059804 2534 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059807 2534 flags.go:64] FLAG: --eviction-hard="" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059810 2534 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059813 2534 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059816 2534 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059819 2534 flags.go:64] FLAG: --eviction-soft="" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059821 2534 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059824 2534 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059829 2534 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059832 2534 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059834 2534 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059837 2534 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059840 2534 flags.go:64] FLAG: --feature-gates="" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059843 2534 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059846 2534 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059849 2534 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059852 2534 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059855 2534 flags.go:64] FLAG: --healthz-port="10248" Jul 09 04:06:50.064936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059858 2534 flags.go:64] FLAG: --help="false" Jul 09 04:06:50.065617 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059861 2534 flags.go:64] FLAG: --hostname-override="ip-10-0-130-153.ec2.internal" Jul 09 04:06:50.065617 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059864 2534 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 04:06:50.065617 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059866 2534 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 04:06:50.065617 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059869 2534 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 04:06:50.065617 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059872 2534 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 04:06:50.065617 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059876 2534 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 04:06:50.065617 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059878 2534 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 04:06:50.065617 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059881 2534 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 04:06:50.065617 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059884 2534 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 04:06:50.065617 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059886 2534 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 04:06:50.065617 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059889 2534 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 04:06:50.065617 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059892 2534 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 04:06:50.065617 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059897 2534 flags.go:64] FLAG: --kube-reserved="" Jul 09 04:06:50.065617 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059900 2534 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 04:06:50.065617 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059903 2534 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 04:06:50.065617 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059906 2534 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 04:06:50.065617 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059908 2534 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 04:06:50.065617 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059911 2534 flags.go:64] FLAG: --lock-file="" Jul 09 04:06:50.065617 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059914 2534 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 04:06:50.065617 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059917 2534 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 04:06:50.065617 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059919 2534 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 04:06:50.065617 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059926 2534 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 04:06:50.065617 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059928 2534 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 04:06:50.066198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059931 2534 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 04:06:50.066198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059934 2534 flags.go:64] FLAG: --logging-format="text" Jul 09 04:06:50.066198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059936 2534 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 04:06:50.066198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059940 2534 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 04:06:50.066198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059942 2534 flags.go:64] FLAG: --manifest-url="" Jul 09 04:06:50.066198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059945 2534 flags.go:64] FLAG: --manifest-url-header="" Jul 09 04:06:50.066198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059949 2534 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 04:06:50.066198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059951 2534 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 04:06:50.066198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059955 2534 flags.go:64] FLAG: --max-pods="110" Jul 09 04:06:50.066198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059959 2534 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 04:06:50.066198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059962 2534 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 04:06:50.066198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059965 2534 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 04:06:50.066198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059967 2534 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 04:06:50.066198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059970 2534 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 04:06:50.066198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059972 2534 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 04:06:50.066198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059975 2534 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 04:06:50.066198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059982 2534 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 04:06:50.066198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059984 2534 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 04:06:50.066198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059987 2534 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 04:06:50.066198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059990 2534 flags.go:64] FLAG: --pod-cidr="" Jul 09 04:06:50.066198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059993 2534 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 09 04:06:50.066198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.059999 2534 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 04:06:50.066198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060001 2534 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 04:06:50.066198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060004 2534 flags.go:64] FLAG: --pods-per-core="0" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060007 2534 flags.go:64] FLAG: --port="10250" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060010 2534 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060013 2534 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0eb75c98b73e3a363" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060016 2534 flags.go:64] FLAG: --qos-reserved="" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060019 2534 flags.go:64] FLAG: --read-only-port="10255" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060021 2534 flags.go:64] FLAG: --register-node="true" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060024 2534 flags.go:64] FLAG: --register-with-taints="" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060028 2534 flags.go:64] FLAG: --registry-burst="10" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060031 2534 flags.go:64] FLAG: --registry-qps="5" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060034 2534 flags.go:64] FLAG: --reserved-cpus="" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060037 2534 flags.go:64] FLAG: --reserved-memory="" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060040 2534 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060043 2534 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060046 2534 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060049 2534 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060052 2534 flags.go:64] FLAG: --runonce="false" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060054 2534 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060057 2534 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060061 2534 flags.go:64] FLAG: --seccomp-default="false" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060064 2534 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060066 2534 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060069 2534 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060072 2534 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060074 2534 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060077 2534 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 04:06:50.066792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060080 2534 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 04:06:50.067418 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060083 2534 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 04:06:50.067418 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060085 2534 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 04:06:50.067418 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060088 2534 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 04:06:50.067418 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060092 2534 flags.go:64] FLAG: --system-cgroups="" Jul 09 04:06:50.067418 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060094 2534 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 04:06:50.067418 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060099 2534 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 04:06:50.067418 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060102 2534 flags.go:64] FLAG: --tls-cert-file="" Jul 09 04:06:50.067418 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060104 2534 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 04:06:50.067418 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060108 2534 flags.go:64] FLAG: --tls-min-version="" Jul 09 04:06:50.067418 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060111 2534 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 04:06:50.067418 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060113 2534 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 04:06:50.067418 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060116 2534 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 04:06:50.067418 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060119 2534 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 04:06:50.067418 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060123 2534 flags.go:64] FLAG: --v="2" Jul 09 04:06:50.067418 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060126 2534 flags.go:64] FLAG: --version="false" Jul 09 04:06:50.067418 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060130 2534 flags.go:64] FLAG: --vmodule="" Jul 09 04:06:50.067418 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060133 2534 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 04:06:50.067418 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.060136 2534 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 04:06:50.067418 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060261 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 04:06:50.067418 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060266 2534 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 04:06:50.067418 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060270 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 04:06:50.067418 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060275 2534 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 04:06:50.067418 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060291 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 04:06:50.067418 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060294 2534 feature_gate.go:328] unrecognized feature gate: Example Jul 09 04:06:50.068008 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060297 2534 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 04:06:50.068008 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060300 2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 04:06:50.068008 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060303 2534 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 04:06:50.068008 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060305 2534 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 04:06:50.068008 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060307 2534 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 04:06:50.068008 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060310 2534 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 04:06:50.068008 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060312 2534 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 04:06:50.068008 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060315 2534 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 04:06:50.068008 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060317 2534 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 04:06:50.068008 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060320 2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 04:06:50.068008 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060322 2534 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 04:06:50.068008 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060326 2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 04:06:50.068008 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060329 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 04:06:50.068008 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060331 2534 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 04:06:50.068008 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060334 2534 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 04:06:50.068008 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060336 2534 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 04:06:50.068008 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060339 2534 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 04:06:50.068008 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060341 2534 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 04:06:50.068008 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060344 2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 04:06:50.068008 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060346 2534 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 04:06:50.068592 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060348 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 04:06:50.068592 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060352 2534 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 04:06:50.068592 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060355 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 04:06:50.068592 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060357 2534 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 04:06:50.068592 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060360 2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 04:06:50.068592 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060362 2534 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 04:06:50.068592 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060366 2534 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 04:06:50.068592 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060369 2534 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 04:06:50.068592 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060372 2534 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 04:06:50.068592 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060375 2534 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 04:06:50.068592 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060379 2534 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 04:06:50.068592 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060381 2534 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 04:06:50.068592 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060384 2534 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 04:06:50.068592 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060387 2534 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 04:06:50.068592 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060390 2534 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 04:06:50.068592 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060393 2534 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 04:06:50.068592 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060395 2534 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 04:06:50.068592 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060398 2534 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 04:06:50.068592 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060400 2534 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 04:06:50.069036 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060403 2534 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 04:06:50.069036 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060405 2534 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 04:06:50.069036 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060408 2534 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 04:06:50.069036 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060410 2534 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 04:06:50.069036 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060413 2534 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 04:06:50.069036 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060416 2534 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 04:06:50.069036 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060418 2534 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 04:06:50.069036 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060421 2534 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 04:06:50.069036 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060423 2534 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 04:06:50.069036 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060426 2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 04:06:50.069036 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060428 2534 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 04:06:50.069036 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060431 2534 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 04:06:50.069036 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060433 2534 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 04:06:50.069036 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060436 2534 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 04:06:50.069036 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060439 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 04:06:50.069036 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060442 2534 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 04:06:50.069036 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060445 2534 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 04:06:50.069036 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060447 2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 04:06:50.069036 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060450 2534 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 04:06:50.069036 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060452 2534 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 04:06:50.069535 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060454 2534 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 04:06:50.069535 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060457 2534 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 04:06:50.069535 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060459 2534 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 04:06:50.069535 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060462 2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 04:06:50.069535 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060464 2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 04:06:50.069535 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060466 2534 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 04:06:50.069535 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060469 2534 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 04:06:50.069535 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060471 2534 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 04:06:50.069535 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060474 2534 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 04:06:50.069535 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060477 2534 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 04:06:50.069535 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060479 2534 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 04:06:50.069535 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060482 2534 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 04:06:50.069535 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060484 2534 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 04:06:50.069535 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060486 2534 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 04:06:50.069535 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060489 2534 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 04:06:50.069535 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060491 2534 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 04:06:50.069535 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060494 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 04:06:50.069535 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060496 2534 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 04:06:50.069535 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060500 2534 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 04:06:50.070002 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060503 2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 04:06:50.070002 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060505 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 04:06:50.070002 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060508 2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 04:06:50.070002 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060511 2534 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 04:06:50.070002 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060514 2534 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 04:06:50.070002 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060517 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 04:06:50.070002 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060519 2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 04:06:50.070002 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060523 2534 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 04:06:50.070002 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060526 2534 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 04:06:50.070002 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060529 2534 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 04:06:50.070002 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060531 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 04:06:50.070002 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060534 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 04:06:50.070002 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060536 2534 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 04:06:50.070002 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060538 2534 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 04:06:50.070002 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.060541 2534 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 04:06:50.070388 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.061098 2534 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 04:06:50.070388 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.067497 2534 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 09 04:06:50.070388 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.067513 2534 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 04:06:50.070388 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.067530 2534 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 04:06:50.070388 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.067536 2534 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 04:06:50.070388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067589 2534 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 04:06:50.070388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067594 2534 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 04:06:50.070388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067597 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 04:06:50.070388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067602 2534 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 04:06:50.070388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067606 2534 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 04:06:50.070388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067609 2534 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 04:06:50.070388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067612 2534 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 04:06:50.070388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067615 2534 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 04:06:50.070388 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067617 2534 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 04:06:50.070747 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067620 2534 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 04:06:50.070747 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067623 2534 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 04:06:50.070747 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067625 2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 04:06:50.070747 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067627 2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 04:06:50.070747 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067630 2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 04:06:50.070747 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067632 2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 04:06:50.070747 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067635 2534 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 04:06:50.070747 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067637 2534 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 04:06:50.070747 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067640 2534 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 04:06:50.070747 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067642 2534 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 04:06:50.070747 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067645 2534 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 04:06:50.070747 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067647 2534 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 04:06:50.070747 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067650 2534 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 04:06:50.070747 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067652 2534 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 04:06:50.070747 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067655 2534 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 04:06:50.070747 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067657 2534 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 04:06:50.070747 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067660 2534 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 04:06:50.070747 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067663 2534 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 04:06:50.070747 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067665 2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 04:06:50.070747 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067668 2534 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 04:06:50.071219 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067670 2534 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 04:06:50.071219 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067673 2534 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 04:06:50.071219 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067675 2534 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 04:06:50.071219 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067678 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 04:06:50.071219 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067680 2534 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 04:06:50.071219 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067683 2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 04:06:50.071219 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067686 2534 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 04:06:50.071219 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067688 2534 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 04:06:50.071219 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067690 2534 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 04:06:50.071219 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067693 2534 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 04:06:50.071219 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067695 2534 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 04:06:50.071219 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067698 2534 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 04:06:50.071219 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067701 2534 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 04:06:50.071219 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067703 2534 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 04:06:50.071219 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067705 2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 04:06:50.071219 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067708 2534 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 04:06:50.071219 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067710 2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 04:06:50.071219 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067712 2534 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 04:06:50.071219 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067715 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 04:06:50.071707 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067717 2534 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 04:06:50.071707 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067720 2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 04:06:50.071707 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067722 2534 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 04:06:50.071707 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067725 2534 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 04:06:50.071707 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067728 2534 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 04:06:50.071707 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067730 2534 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 04:06:50.071707 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067733 2534 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 04:06:50.071707 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067736 2534 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 04:06:50.071707 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067751 2534 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 04:06:50.071707 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067754 2534 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 04:06:50.071707 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067757 2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 04:06:50.071707 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067760 2534 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 04:06:50.071707 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067763 2534 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 04:06:50.071707 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067766 2534 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 04:06:50.071707 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067769 2534 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 04:06:50.071707 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067771 2534 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 04:06:50.071707 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067774 2534 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 04:06:50.071707 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067777 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 04:06:50.071707 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067779 2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 04:06:50.071707 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067782 2534 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 04:06:50.072194 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067785 2534 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 04:06:50.072194 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067787 2534 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 04:06:50.072194 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067790 2534 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 04:06:50.072194 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067792 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 04:06:50.072194 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067795 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 04:06:50.072194 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067797 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 04:06:50.072194 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067799 2534 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 04:06:50.072194 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067802 2534 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 04:06:50.072194 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067804 2534 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 04:06:50.072194 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067806 2534 feature_gate.go:328] unrecognized feature gate: Example Jul 09 04:06:50.072194 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067809 2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 04:06:50.072194 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067811 2534 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 04:06:50.072194 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067814 2534 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 04:06:50.072194 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067816 2534 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 04:06:50.072194 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067819 2534 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 04:06:50.072194 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067822 2534 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 04:06:50.072194 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067824 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 04:06:50.072194 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067827 2534 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 04:06:50.072194 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067829 2534 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 04:06:50.072194 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067831 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 04:06:50.072696 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067834 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 04:06:50.072696 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067838 2534 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 04:06:50.072696 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067841 2534 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 04:06:50.072696 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067844 2534 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 04:06:50.072696 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067847 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 04:06:50.072696 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067850 2534 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 04:06:50.072696 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067852 2534 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 04:06:50.072696 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067854 2534 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 04:06:50.072696 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067857 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 04:06:50.072696 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067859 2534 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 04:06:50.072696 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067863 2534 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 04:06:50.072696 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.067868 2534 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 04:06:50.072696 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067974 2534 feature_gate.go:328] unrecognized feature gate: Example Jul 09 04:06:50.072696 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067978 2534 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 04:06:50.073041 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067980 2534 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 04:06:50.073041 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067984 2534 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 04:06:50.073041 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067986 2534 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 04:06:50.073041 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067989 2534 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 04:06:50.073041 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067991 2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 04:06:50.073041 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067994 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 04:06:50.073041 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.067996 2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 04:06:50.073041 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068000 2534 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 04:06:50.073041 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068003 2534 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 04:06:50.073041 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068006 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 04:06:50.073041 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068009 2534 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 04:06:50.073041 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068012 2534 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 04:06:50.073041 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068014 2534 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 04:06:50.073041 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068017 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 04:06:50.073041 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068020 2534 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 04:06:50.073041 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068022 2534 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 04:06:50.073041 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068025 2534 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 04:06:50.073041 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068027 2534 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 04:06:50.073041 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068030 2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 04:06:50.073521 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068032 2534 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 04:06:50.073521 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068035 2534 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 04:06:50.073521 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068037 2534 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 04:06:50.073521 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068040 2534 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 04:06:50.073521 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068042 2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 04:06:50.073521 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068046 2534 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 04:06:50.073521 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068049 2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 04:06:50.073521 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068051 2534 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 04:06:50.073521 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068054 2534 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 04:06:50.073521 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068057 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 04:06:50.073521 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068059 2534 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 04:06:50.073521 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068061 2534 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 04:06:50.073521 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068064 2534 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 04:06:50.073521 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068067 2534 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 04:06:50.073521 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068069 2534 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 04:06:50.073521 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068071 2534 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 04:06:50.073521 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068074 2534 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 04:06:50.073521 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068076 2534 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 04:06:50.073521 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068078 2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 04:06:50.073521 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068081 2534 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 04:06:50.073995 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068083 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 04:06:50.073995 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068085 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 04:06:50.073995 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068088 2534 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 04:06:50.073995 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068090 2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 04:06:50.073995 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068092 2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 04:06:50.073995 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068095 2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 04:06:50.073995 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068098 2534 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 04:06:50.073995 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068100 2534 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 04:06:50.073995 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068103 2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 04:06:50.073995 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068105 2534 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 04:06:50.073995 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068108 2534 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 04:06:50.073995 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068110 2534 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 04:06:50.073995 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068112 2534 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 04:06:50.073995 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068115 2534 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 04:06:50.073995 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068117 2534 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 04:06:50.073995 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068120 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 04:06:50.073995 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068122 2534 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 04:06:50.073995 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068125 2534 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 04:06:50.073995 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068127 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 04:06:50.073995 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068130 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 04:06:50.074539 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068132 2534 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 04:06:50.074539 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068135 2534 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 04:06:50.074539 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068138 2534 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 04:06:50.074539 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068141 2534 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 04:06:50.074539 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068144 2534 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 04:06:50.074539 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068147 2534 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 04:06:50.074539 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068150 2534 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 04:06:50.074539 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068152 2534 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 04:06:50.074539 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068154 2534 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 04:06:50.074539 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068157 2534 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 04:06:50.074539 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068159 2534 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 04:06:50.074539 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068161 2534 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 04:06:50.074539 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068164 2534 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 04:06:50.074539 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068166 2534 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 04:06:50.074539 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068168 2534 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 04:06:50.074539 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068171 2534 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 04:06:50.074539 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068173 2534 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 04:06:50.074539 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068176 2534 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 04:06:50.074539 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068178 2534 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 04:06:50.075242 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068181 2534 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 04:06:50.075242 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068183 2534 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 04:06:50.075242 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068186 2534 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 04:06:50.075242 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068188 2534 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 04:06:50.075242 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068190 2534 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 04:06:50.075242 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068193 2534 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 04:06:50.075242 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068195 2534 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 04:06:50.075242 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068197 2534 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 04:06:50.075242 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068200 2534 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 04:06:50.075242 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068202 2534 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 04:06:50.075242 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068205 2534 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 04:06:50.075242 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068207 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 04:06:50.075242 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068210 2534 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 04:06:50.075242 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068212 2534 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 04:06:50.075242 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068214 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 04:06:50.075242 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068217 2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 04:06:50.075242 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068219 2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 04:06:50.075242 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068222 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 04:06:50.075242 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:50.068224 2534 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 04:06:50.075775 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.068239 2534 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 04:06:50.075775 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.068354 2534 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 04:06:50.075775 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.070186 2534 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 04:06:50.075775 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.071448 2534 server.go:1019] "Starting client certificate rotation" Jul 09 04:06:50.075775 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.071543 2534 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 04:06:50.075775 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.071578 2534 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 04:06:50.075775 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.074343 2534 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 04:06:50.075775 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.075418 2534 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 04:06:50.087120 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.087100 2534 log.go:25] "Validated CRI v1 runtime API" Jul 09 04:06:50.088935 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.088921 2534 log.go:25] "Validated CRI v1 image API" Jul 09 04:06:50.089668 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.089650 2534 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 04:06:50.092496 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.092478 2534 fs.go:135] Filesystem UUIDs: map[0cb62f7e-1c7a-4eee-8a62-678b20a58b7b:/dev/nvme0n1p3 3e6dc67e-e018-4816-8756-2ca37b7638e3:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 09 04:06:50.092554 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.092496 2534 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 04:06:50.098238 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.098220 2534 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 04:06:50.098381 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.098248 2534 manager.go:216] Machine: {Timestamp:2026-07-09 04:06:50.096996629 +0000 UTC m=+0.320615107 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101546 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec218e3fa9874534106737b02fd4d86f SystemUUID:ec218e3f-a987-4534-1067-37b02fd4d86f BootID:55eb74c8-7fb7-4dec-b331-6249d888cd5b Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e2:ab:2f:dc:19 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e2:ab:2f:dc:19 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:da:54:30:bd:17:81 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 04:06:50.098381 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.098382 2534 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 04:06:50.098472 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.098457 2534 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 04:06:50.101029 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.101006 2534 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 04:06:50.101174 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.101033 2534 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-153.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 04:06:50.101676 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.101666 2534 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 04:06:50.101715 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.101679 2534 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 04:06:50.101715 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.101686 2534 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 04:06:50.102200 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.102191 2534 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 04:06:50.103164 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.103154 2534 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 04:06:50.106678 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.106666 2534 state_mem.go:36] "Initialized new in-memory state store" Jul 09 04:06:50.106793 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.106784 2534 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 04:06:50.106823 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.106813 2534 kubelet.go:486] "Attempting to sync node with API server" Jul 09 04:06:50.106855 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.106823 2534 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 04:06:50.106855 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.106833 2534 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 04:06:50.106855 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.106841 2534 kubelet.go:398] "Adding apiserver pod source" Jul 09 04:06:50.106855 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.106850 2534 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 04:06:50.107236 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.107225 2534 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 04:06:50.107270 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.107243 2534 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 04:06:50.113576 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.113549 2534 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 09 04:06:50.115193 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.115180 2534 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 04:06:50.115233 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.115210 2534 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 04:06:50.116097 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.116085 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 04:06:50.116157 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.116101 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 04:06:50.116157 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.116107 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 04:06:50.116157 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.116113 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 04:06:50.116157 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.116121 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 04:06:50.116157 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.116126 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 04:06:50.116157 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.116133 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 04:06:50.116157 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.116138 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 04:06:50.116157 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.116145 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 04:06:50.116157 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.116150 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 04:06:50.116157 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.116158 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 04:06:50.116157 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.116163 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 04:06:50.116882 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.116869 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 04:06:50.116882 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.116883 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 04:06:50.118755 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.118736 2534 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-153.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 04:06:50.118835 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:50.118778 2534 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-153.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 04:06:50.118835 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:50.118795 2534 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 04:06:50.120050 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.120036 2534 server.go:1268] "Started kubelet" Jul 09 04:06:50.120141 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.120115 2534 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 04:06:50.120619 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.120328 2534 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 04:06:50.120619 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.120402 2534 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 04:06:50.120900 ip-10-0-130-153 systemd[1]: Started Kubernetes Kubelet. Jul 09 04:06:50.121448 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.121349 2534 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 04:06:50.125161 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.125143 2534 server.go:310] "Adding debug handlers to kubelet server" Jul 09 04:06:50.125365 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:50.125341 2534 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 04:06:50.128328 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:50.126360 2534 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-153.ec2.internal.18c08379753e300b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-153.ec2.internal,UID:ip-10-0-130-153.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-153.ec2.internal,},FirstTimestamp:2026-07-09 04:06:50.120007691 +0000 UTC m=+0.343626172,LastTimestamp:2026-07-09 04:06:50.120007691 +0000 UTC m=+0.343626172,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-153.ec2.internal,}" Jul 09 04:06:50.131212 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.131196 2534 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 04:06:50.131302 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.131215 2534 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 04:06:50.131302 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:50.131294 2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-153.ec2.internal\" not found" Jul 09 04:06:50.131735 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.131715 2534 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 04:06:50.131735 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.131721 2534 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 04:06:50.131879 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.131741 2534 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 04:06:50.131879 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.131826 2534 reconstruct.go:97] "Volume reconstruction finished" Jul 09 04:06:50.131879 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.131855 2534 reconciler.go:29] "Reconciler: start to sync state" Jul 09 04:06:50.133991 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.133975 2534 factory.go:55] Registering systemd factory Jul 09 04:06:50.133991 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.133994 2534 factory.go:223] Registration of the systemd container factory successfully Jul 09 04:06:50.134301 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.134259 2534 factory.go:153] Registering CRI-O factory Jul 09 04:06:50.134383 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.134306 2534 factory.go:223] Registration of the crio container factory successfully Jul 09 04:06:50.134383 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.134365 2534 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 04:06:50.134489 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.134395 2534 factory.go:103] Registering Raw factory Jul 09 04:06:50.134489 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.134410 2534 manager.go:1196] Started watching for new ooms in manager Jul 09 04:06:50.135038 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.135022 2534 manager.go:349] Starting recovery of all containers Jul 09 04:06:50.141446 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:50.141420 2534 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 04:06:50.141513 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:50.141455 2534 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-153.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 04:06:50.142116 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.142093 2534 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xxkgq" Jul 09 04:06:50.144655 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.144488 2534 manager.go:354] Recovery completed Jul 09 04:06:50.148720 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.148707 2534 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 04:06:50.149486 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.149470 2534 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xxkgq" Jul 09 04:06:50.150632 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.150617 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-153.ec2.internal" event="NodeHasSufficientMemory" Jul 09 04:06:50.150698 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.150644 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-153.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 04:06:50.150698 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.150654 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-153.ec2.internal" event="NodeHasSufficientPID" Jul 09 04:06:50.151190 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.151176 2534 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 04:06:50.151190 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.151186 2534 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 04:06:50.151347 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.151189 2534 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 04:06:50.151347 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.151229 2534 state_mem.go:36] "Initialized new in-memory state store" Jul 09 04:06:50.151996 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.151983 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-153.ec2.internal" event="NodeHasSufficientMemory" Jul 09 04:06:50.152044 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.152012 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-153.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 04:06:50.152044 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.152025 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-153.ec2.internal" event="NodeHasSufficientPID" Jul 09 04:06:50.153343 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.153332 2534 policy_none.go:49] "None policy: Start" Jul 09 04:06:50.153388 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.153347 2534 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 04:06:50.153388 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.153357 2534 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 04:06:50.155241 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.155229 2534 policy_none.go:47] "Start" Jul 09 04:06:50.186506 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.186487 2534 manager.go:337] "Starting Device Plugin manager" Jul 09 04:06:50.213632 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:50.186516 2534 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 04:06:50.213632 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.186526 2534 server.go:87] "Starting device plugin registration server" Jul 09 04:06:50.213632 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.186935 2534 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 04:06:50.213632 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.186948 2534 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 04:06:50.213632 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.187064 2534 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 04:06:50.213632 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.187135 2534 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 04:06:50.213632 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.187143 2534 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 04:06:50.213632 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:50.187607 2534 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 04:06:50.213632 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:50.187656 2534 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-153.ec2.internal\" not found" Jul 09 04:06:50.242490 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.242460 2534 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 04:06:50.243698 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.243669 2534 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 04:06:50.243770 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.243727 2534 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 04:06:50.243770 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.243756 2534 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 04:06:50.243849 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:50.243796 2534 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 04:06:50.246111 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.246093 2534 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 04:06:50.287153 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.287136 2534 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 04:06:50.287899 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.287884 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-153.ec2.internal" event="NodeHasSufficientMemory" Jul 09 04:06:50.287979 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.287912 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-153.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 04:06:50.287979 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.287922 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-153.ec2.internal" event="NodeHasSufficientPID" Jul 09 04:06:50.287979 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.287947 2534 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-153.ec2.internal" Jul 09 04:06:50.299603 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.299586 2534 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-153.ec2.internal" Jul 09 04:06:50.299654 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:50.299605 2534 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-153.ec2.internal\": node \"ip-10-0-130-153.ec2.internal\" not found" Jul 09 04:06:50.323679 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:50.323622 2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-153.ec2.internal\" not found" Jul 09 04:06:50.344163 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.344136 2534 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-153.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-153.ec2.internal"] Jul 09 04:06:50.344247 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.344206 2534 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 04:06:50.349159 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.349144 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-153.ec2.internal" event="NodeHasSufficientMemory" Jul 09 04:06:50.349269 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.349167 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-153.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 04:06:50.349269 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.349176 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-153.ec2.internal" event="NodeHasSufficientPID" Jul 09 04:06:50.351584 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.351570 2534 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 04:06:50.351687 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.351673 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-153.ec2.internal" Jul 09 04:06:50.351722 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.351701 2534 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 04:06:50.353697 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.353681 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-153.ec2.internal" event="NodeHasSufficientMemory" Jul 09 04:06:50.353697 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.353692 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-153.ec2.internal" event="NodeHasSufficientMemory" Jul 09 04:06:50.353858 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.353712 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-153.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 04:06:50.353858 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.353727 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-153.ec2.internal" event="NodeHasSufficientPID" Jul 09 04:06:50.353858 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.353714 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-153.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 04:06:50.353858 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.353838 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-153.ec2.internal" event="NodeHasSufficientPID" Jul 09 04:06:50.356017 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.356002 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-153.ec2.internal" Jul 09 04:06:50.356109 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.356025 2534 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 04:06:50.356686 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.356673 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-153.ec2.internal" event="NodeHasSufficientMemory" Jul 09 04:06:50.356761 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.356697 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-153.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 04:06:50.356761 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.356710 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-153.ec2.internal" event="NodeHasSufficientPID" Jul 09 04:06:50.375863 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:50.375835 2534 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-153.ec2.internal\" not found" node="ip-10-0-130-153.ec2.internal" Jul 09 04:06:50.379156 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:50.379138 2534 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-153.ec2.internal\" not found" node="ip-10-0-130-153.ec2.internal" Jul 09 04:06:50.424080 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:50.424061 2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-153.ec2.internal\" not found" Jul 09 04:06:50.433795 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.433778 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e28bfc992df73a71242390a6bf756269-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-153.ec2.internal\" (UID: \"e28bfc992df73a71242390a6bf756269\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-153.ec2.internal" Jul 09 04:06:50.433849 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.433803 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e28bfc992df73a71242390a6bf756269-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-153.ec2.internal\" (UID: \"e28bfc992df73a71242390a6bf756269\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-153.ec2.internal" Jul 09 04:06:50.433849 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.433821 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/67b85ef8254593a9d9a03d6a7f9822a9-config\") pod \"kube-apiserver-proxy-ip-10-0-130-153.ec2.internal\" (UID: \"67b85ef8254593a9d9a03d6a7f9822a9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-153.ec2.internal" Jul 09 04:06:50.524546 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:50.524516 2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-153.ec2.internal\" not found" Jul 09 04:06:50.534932 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.534915 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/67b85ef8254593a9d9a03d6a7f9822a9-config\") pod \"kube-apiserver-proxy-ip-10-0-130-153.ec2.internal\" (UID: \"67b85ef8254593a9d9a03d6a7f9822a9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-153.ec2.internal" Jul 09 04:06:50.534975 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.534943 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/67b85ef8254593a9d9a03d6a7f9822a9-config\") pod \"kube-apiserver-proxy-ip-10-0-130-153.ec2.internal\" (UID: \"67b85ef8254593a9d9a03d6a7f9822a9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-153.ec2.internal" Jul 09 04:06:50.534975 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.534954 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e28bfc992df73a71242390a6bf756269-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-153.ec2.internal\" (UID: \"e28bfc992df73a71242390a6bf756269\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-153.ec2.internal" Jul 09 04:06:50.534975 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.534973 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e28bfc992df73a71242390a6bf756269-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-153.ec2.internal\" (UID: \"e28bfc992df73a71242390a6bf756269\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-153.ec2.internal" Jul 09 04:06:50.535069 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.534986 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e28bfc992df73a71242390a6bf756269-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-153.ec2.internal\" (UID: \"e28bfc992df73a71242390a6bf756269\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-153.ec2.internal" Jul 09 04:06:50.535069 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.535011 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e28bfc992df73a71242390a6bf756269-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-153.ec2.internal\" (UID: \"e28bfc992df73a71242390a6bf756269\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-153.ec2.internal" Jul 09 04:06:50.625408 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:50.625339 2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-153.ec2.internal\" not found" Jul 09 04:06:50.678787 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.678766 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-153.ec2.internal" Jul 09 04:06:50.681933 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:50.681917 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-153.ec2.internal" Jul 09 04:06:50.725955 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:50.725926 2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-153.ec2.internal\" not found" Jul 09 04:06:50.826439 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:50.826397 2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-153.ec2.internal\" not found" Jul 09 04:06:50.927023 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:50.926939 2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-153.ec2.internal\" not found" Jul 09 04:06:51.027426 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:51.027398 2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-153.ec2.internal\" not found" Jul 09 04:06:51.070964 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:51.070940 2534 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 04:06:51.071442 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:51.071074 2534 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 04:06:51.127975 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:51.127954 2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-153.ec2.internal\" not found" Jul 09 04:06:51.154714 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:51.154677 2534 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 04:01:51 +0000 UTC" deadline="2027-12-11 19:37:20.800106334 +0000 UTC" Jul 09 04:06:51.154714 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:51.154710 2534 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12495h30m29.645400417s" Jul 09 04:06:51.186159 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:51.186107 2534 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 04:06:51.212377 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:51.212355 2534 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 04:06:51.231924 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:51.231900 2534 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-153.ec2.internal" Jul 09 04:06:51.244540 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:51.244522 2534 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 04:06:51.245533 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:51.245515 2534 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-153.ec2.internal" Jul 09 04:06:51.254534 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:51.254520 2534 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 04:06:51.381219 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:51.381196 2534 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 04:06:51.383488 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:51.383452 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b85ef8254593a9d9a03d6a7f9822a9.slice/crio-863ba7342a980a1197b393904943298e94aa2bbae36bc2260048e389e98746c3 WatchSource:0}: Error finding container 863ba7342a980a1197b393904943298e94aa2bbae36bc2260048e389e98746c3: Status 404 returned error can't find the container with id 863ba7342a980a1197b393904943298e94aa2bbae36bc2260048e389e98746c3 Jul 09 04:06:51.383894 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:51.383873 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode28bfc992df73a71242390a6bf756269.slice/crio-eb880e83114428ffa6caa0898fc537a39a33d9aaa1f3d84740316e21ceee2a4c WatchSource:0}: Error finding container eb880e83114428ffa6caa0898fc537a39a33d9aaa1f3d84740316e21ceee2a4c: Status 404 returned error can't find the container with id eb880e83114428ffa6caa0898fc537a39a33d9aaa1f3d84740316e21ceee2a4c Jul 09 04:06:51.388166 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:51.388152 2534 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 04:06:52.046292 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.046261 2534 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 04:06:52.107705 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.107660 2534 apiserver.go:52] "Watching apiserver" Jul 09 04:06:52.120891 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.120305 2534 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 04:06:52.122232 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.122200 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-153.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf","openshift-cluster-node-tuning-operator/tuned-ks45b","openshift-image-registry/node-ca-dgz8s","openshift-network-diagnostics/network-check-target-8xpfm","openshift-ovn-kubernetes/ovnkube-node-pkb8q","openshift-dns/node-resolver-zpr9q","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-153.ec2.internal","openshift-multus/multus-additional-cni-plugins-k9th4","openshift-multus/multus-bv4mn","openshift-multus/network-metrics-daemon-b4gcd","openshift-network-operator/iptables-alerter-rh79z","kube-system/global-pull-secret-syncer-krxxf","kube-system/konnectivity-agent-xksp2"] Jul 09 04:06:52.125066 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.125044 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zpr9q" Jul 09 04:06:52.127481 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.127458 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:06:52.127598 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:52.127539 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krxxf" podUID="2c0aaa25-8a07-4da9-8d7f-b8abd6026f36" Jul 09 04:06:52.127734 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.127704 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6wzsb\"" Jul 09 04:06:52.127855 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.127837 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 04:06:52.127915 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.127838 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 04:06:52.131221 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.131200 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.132316 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.132138 2534 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 04:06:52.132316 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.132262 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dgz8s" Jul 09 04:06:52.133405 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.133344 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-229w2\"" Jul 09 04:06:52.133600 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.133582 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 04:06:52.133821 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.133801 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 04:06:52.134640 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.134295 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 04:06:52.135529 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.134955 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 04:06:52.135529 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.135006 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vkghk\"" Jul 09 04:06:52.135529 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.135061 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:06:52.135529 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:52.135126 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xpfm" podUID="8e433de1-ad41-4547-b130-e06490dba1bb" Jul 09 04:06:52.135529 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.135201 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 04:06:52.139687 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.138405 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.140655 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.140634 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.140888 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.140855 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 04:06:52.140990 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.140918 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gfqvq\"" Jul 09 04:06:52.141208 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.141191 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 04:06:52.141344 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.141286 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 04:06:52.142639 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.142620 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 04:06:52.142737 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.142713 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 04:06:52.142929 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.142905 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-run\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.143008 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.142940 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-host\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.143008 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.142969 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5wq8\" (UniqueName: \"kubernetes.io/projected/758a58c5-cb89-447a-b25a-6c3d87b1ba11-kube-api-access-p5wq8\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.143008 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.142983 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 04:06:52.143008 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.142996 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8104c5b9-011c-4e10-9a15-53bdd52870f0-hosts-file\") pod \"node-resolver-zpr9q\" (UID: \"8104c5b9-011c-4e10-9a15-53bdd52870f0\") " pod="openshift-dns/node-resolver-zpr9q" Jul 09 04:06:52.143207 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.143015 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-dbus\") pod \"global-pull-secret-syncer-krxxf\" (UID: \"2c0aaa25-8a07-4da9-8d7f-b8abd6026f36\") " pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:06:52.143207 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.143035 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-etc-kubernetes\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.143207 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.143058 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-etc-sysctl-d\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.143207 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.143092 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-sys\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.143207 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.143113 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-var-lib-kubelet\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.143207 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.143136 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/758a58c5-cb89-447a-b25a-6c3d87b1ba11-tmp\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.143546 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.143226 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0a7d54c6-9262-4462-8b4b-f35a678e2ddf-host\") pod \"node-ca-dgz8s\" (UID: \"0a7d54c6-9262-4462-8b4b-f35a678e2ddf\") " pod="openshift-image-registry/node-ca-dgz8s" Jul 09 04:06:52.143546 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.143259 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0a7d54c6-9262-4462-8b4b-f35a678e2ddf-serviceca\") pod \"node-ca-dgz8s\" (UID: \"0a7d54c6-9262-4462-8b4b-f35a678e2ddf\") " pod="openshift-image-registry/node-ca-dgz8s" Jul 09 04:06:52.143546 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.143284 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8104c5b9-011c-4e10-9a15-53bdd52870f0-tmp-dir\") pod \"node-resolver-zpr9q\" (UID: \"8104c5b9-011c-4e10-9a15-53bdd52870f0\") " pod="openshift-dns/node-resolver-zpr9q" Jul 09 04:06:52.143546 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.143315 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-kubelet-config\") pod \"global-pull-secret-syncer-krxxf\" (UID: \"2c0aaa25-8a07-4da9-8d7f-b8abd6026f36\") " pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:06:52.143546 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.143364 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-etc-systemd\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.143546 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.143393 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmpnc\" (UniqueName: \"kubernetes.io/projected/8104c5b9-011c-4e10-9a15-53bdd52870f0-kube-api-access-hmpnc\") pod \"node-resolver-zpr9q\" (UID: \"8104c5b9-011c-4e10-9a15-53bdd52870f0\") " pod="openshift-dns/node-resolver-zpr9q" Jul 09 04:06:52.143546 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.143418 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-etc-modprobe-d\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.143546 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.143490 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 04:06:52.143546 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.143510 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-lib-modules\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.143546 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.143538 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/758a58c5-cb89-447a-b25a-6c3d87b1ba11-etc-tuned\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.144045 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.143573 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-original-pull-secret\") pod \"global-pull-secret-syncer-krxxf\" (UID: \"2c0aaa25-8a07-4da9-8d7f-b8abd6026f36\") " pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:06:52.144045 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.143598 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-etc-sysconfig\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.144045 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.143622 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2nmj\" (UniqueName: \"kubernetes.io/projected/0a7d54c6-9262-4462-8b4b-f35a678e2ddf-kube-api-access-m2nmj\") pod \"node-ca-dgz8s\" (UID: \"0a7d54c6-9262-4462-8b4b-f35a678e2ddf\") " pod="openshift-image-registry/node-ca-dgz8s" Jul 09 04:06:52.144045 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.143666 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-etc-sysctl-conf\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.144045 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.143778 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xksp2" Jul 09 04:06:52.145146 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.144483 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 04:06:52.145146 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.144535 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6pml2\"" Jul 09 04:06:52.145146 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.144560 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 04:06:52.145146 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.144912 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 04:06:52.145146 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.144969 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 04:06:52.145146 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.145077 2534 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 04:06:52.146673 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.145870 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" Jul 09 04:06:52.146673 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.145927 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 04:06:52.146673 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.146514 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-66hzb\"" Jul 09 04:06:52.147127 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.147111 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 04:06:52.148404 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.148351 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 04:06:52.148511 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.148494 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7x4hc\"" Jul 09 04:06:52.149074 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.148783 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.149074 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.148803 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 04:06:52.149074 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.148860 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 04:06:52.150937 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.150915 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 04:06:52.151100 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.151075 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:06:52.151176 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:52.151138 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4gcd" podUID="9fefcefc-4128-4422-98de-b17e5949cfc7" Jul 09 04:06:52.151237 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.151178 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nc5cw\"" Jul 09 04:06:52.153511 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.153493 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rh79z" Jul 09 04:06:52.157127 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.156056 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 04:06:52.157127 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.156108 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 04:06:52.157127 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.156394 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 04:06:52.157127 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.157002 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qlm5q\"" Jul 09 04:06:52.168023 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.168005 2534 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-n79xc" Jul 09 04:06:52.173691 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.173673 2534 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-n79xc" Jul 09 04:06:52.233668 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.233646 2534 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 04:06:52.244650 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.244620 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0a7d54c6-9262-4462-8b4b-f35a678e2ddf-host\") pod \"node-ca-dgz8s\" (UID: \"0a7d54c6-9262-4462-8b4b-f35a678e2ddf\") " pod="openshift-image-registry/node-ca-dgz8s" Jul 09 04:06:52.244760 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.244656 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8104c5b9-011c-4e10-9a15-53bdd52870f0-tmp-dir\") pod \"node-resolver-zpr9q\" (UID: \"8104c5b9-011c-4e10-9a15-53bdd52870f0\") " pod="openshift-dns/node-resolver-zpr9q" Jul 09 04:06:52.244760 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.244674 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-etc-modprobe-d\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.244760 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.244689 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-lib-modules\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.244760 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.244708 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7867a42f-0f05-4c3c-8e48-fa68962734c1-host-slash\") pod \"iptables-alerter-rh79z\" (UID: \"7867a42f-0f05-4c3c-8e48-fa68962734c1\") " pod="openshift-network-operator/iptables-alerter-rh79z" Jul 09 04:06:52.244760 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.244725 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-run-systemd\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.244760 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.244741 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-run-ovn\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.245088 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.244766 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4b157285-2103-4dc1-b461-ac80ec29eb97-ovn-node-metrics-cert\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.245088 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.244782 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee4f80f2-7676-489e-9fe0-99c52e384790-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k9th4\" (UID: \"ee4f80f2-7676-489e-9fe0-99c52e384790\") " pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.245088 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.244797 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9d4cab2f-0478-4c53-9966-d96d5e18f2dd-sys-fs\") pod \"aws-ebs-csi-driver-node-58mgf\" (UID: \"9d4cab2f-0478-4c53-9966-d96d5e18f2dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" Jul 09 04:06:52.245088 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.244813 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-original-pull-secret\") pod \"global-pull-secret-syncer-krxxf\" (UID: \"2c0aaa25-8a07-4da9-8d7f-b8abd6026f36\") " pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:06:52.245088 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.244829 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-etc-sysctl-conf\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.245088 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.244849 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-run\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.245088 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.244898 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-systemd-units\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.245088 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.244936 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-etc-openvswitch\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.245088 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.244957 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-node-log\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.245088 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.244991 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/889db0de-781d-43d6-9df1-7a9eb15a1794-cni-binary-copy\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.245088 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245016 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9d4cab2f-0478-4c53-9966-d96d5e18f2dd-etc-selinux\") pod \"aws-ebs-csi-driver-node-58mgf\" (UID: \"9d4cab2f-0478-4c53-9966-d96d5e18f2dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" Jul 09 04:06:52.245088 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245049 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8104c5b9-011c-4e10-9a15-53bdd52870f0-hosts-file\") pod \"node-resolver-zpr9q\" (UID: \"8104c5b9-011c-4e10-9a15-53bdd52870f0\") " pod="openshift-dns/node-resolver-zpr9q" Jul 09 04:06:52.245088 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245075 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-etc-kubernetes\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.245726 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245099 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-host-run-netns\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.245726 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245129 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-host-cni-bin\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.245726 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245154 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-multus-socket-dir-parent\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.245726 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245188 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-log-socket\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.245726 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245214 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee4f80f2-7676-489e-9fe0-99c52e384790-system-cni-dir\") pod \"multus-additional-cni-plugins-k9th4\" (UID: \"ee4f80f2-7676-489e-9fe0-99c52e384790\") " pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.245726 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245237 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e6cff921-2b0d-4748-ab0f-642fc9ff3d7e-agent-certs\") pod \"konnectivity-agent-xksp2\" (UID: \"e6cff921-2b0d-4748-ab0f-642fc9ff3d7e\") " pod="kube-system/konnectivity-agent-xksp2" Jul 09 04:06:52.245726 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245262 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9d4cab2f-0478-4c53-9966-d96d5e18f2dd-registration-dir\") pod \"aws-ebs-csi-driver-node-58mgf\" (UID: \"9d4cab2f-0478-4c53-9966-d96d5e18f2dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" Jul 09 04:06:52.245726 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245334 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-run-openvswitch\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.245726 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245373 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0a7d54c6-9262-4462-8b4b-f35a678e2ddf-host\") pod \"node-ca-dgz8s\" (UID: \"0a7d54c6-9262-4462-8b4b-f35a678e2ddf\") " pod="openshift-image-registry/node-ca-dgz8s" Jul 09 04:06:52.245726 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245380 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-run\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.245726 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245376 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-host-var-lib-kubelet\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.245726 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245434 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ee4f80f2-7676-489e-9fe0-99c52e384790-os-release\") pod \"multus-additional-cni-plugins-k9th4\" (UID: \"ee4f80f2-7676-489e-9fe0-99c52e384790\") " pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.245726 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245487 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-etc-kubernetes\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.245726 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245559 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-etc-modprobe-d\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.245726 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245589 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-etc-sysctl-conf\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.245726 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245598 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs\") pod \"network-metrics-daemon-b4gcd\" (UID: \"9fefcefc-4128-4422-98de-b17e5949cfc7\") " pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:06:52.245726 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:52.245601 2534 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 04:06:52.246597 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245608 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-lib-modules\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.246597 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245625 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-host\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.246597 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245657 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8104c5b9-011c-4e10-9a15-53bdd52870f0-hosts-file\") pod \"node-resolver-zpr9q\" (UID: \"8104c5b9-011c-4e10-9a15-53bdd52870f0\") " pod="openshift-dns/node-resolver-zpr9q" Jul 09 04:06:52.246597 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245663 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-host-kubelet\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.246597 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245695 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-host\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.246597 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:52.245703 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-original-pull-secret podName:2c0aaa25-8a07-4da9-8d7f-b8abd6026f36 nodeName:}" failed. No retries permitted until 2026-07-09 04:06:52.7456593 +0000 UTC m=+2.969277783 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-original-pull-secret") pod "global-pull-secret-syncer-krxxf" (UID: "2c0aaa25-8a07-4da9-8d7f-b8abd6026f36") : object "kube-system"/"original-pull-secret" not registered Jul 09 04:06:52.246597 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245749 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8104c5b9-011c-4e10-9a15-53bdd52870f0-tmp-dir\") pod \"node-resolver-zpr9q\" (UID: \"8104c5b9-011c-4e10-9a15-53bdd52870f0\") " pod="openshift-dns/node-resolver-zpr9q" Jul 09 04:06:52.246597 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245756 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-system-cni-dir\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.246597 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245828 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ee4f80f2-7676-489e-9fe0-99c52e384790-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k9th4\" (UID: \"ee4f80f2-7676-489e-9fe0-99c52e384790\") " pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.246597 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245865 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-sys\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.246597 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245880 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/758a58c5-cb89-447a-b25a-6c3d87b1ba11-tmp\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.246597 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245896 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4b157285-2103-4dc1-b461-ac80ec29eb97-env-overrides\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.246597 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245912 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-host-run-netns\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.246597 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245942 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-host-var-lib-cni-multus\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.246597 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245969 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-multus-conf-dir\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.246597 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245973 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-sys\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.246597 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.245995 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/889db0de-781d-43d6-9df1-7a9eb15a1794-multus-daemon-config\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.247569 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246024 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e6cff921-2b0d-4748-ab0f-642fc9ff3d7e-konnectivity-ca\") pod \"konnectivity-agent-xksp2\" (UID: \"e6cff921-2b0d-4748-ab0f-642fc9ff3d7e\") " pod="kube-system/konnectivity-agent-xksp2" Jul 09 04:06:52.247569 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246054 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0a7d54c6-9262-4462-8b4b-f35a678e2ddf-serviceca\") pod \"node-ca-dgz8s\" (UID: \"0a7d54c6-9262-4462-8b4b-f35a678e2ddf\") " pod="openshift-image-registry/node-ca-dgz8s" Jul 09 04:06:52.247569 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246078 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-etc-systemd\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.247569 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246103 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-var-lib-openvswitch\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.247569 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246128 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4b157285-2103-4dc1-b461-ac80ec29eb97-ovnkube-config\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.247569 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246151 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56z2j\" (UniqueName: \"kubernetes.io/projected/4b157285-2103-4dc1-b461-ac80ec29eb97-kube-api-access-56z2j\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.247569 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246168 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-etc-systemd\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.247569 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246169 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9d4cab2f-0478-4c53-9966-d96d5e18f2dd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-58mgf\" (UID: \"9d4cab2f-0478-4c53-9966-d96d5e18f2dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" Jul 09 04:06:52.247569 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246218 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9d4cab2f-0478-4c53-9966-d96d5e18f2dd-device-dir\") pod \"aws-ebs-csi-driver-node-58mgf\" (UID: \"9d4cab2f-0478-4c53-9966-d96d5e18f2dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" Jul 09 04:06:52.247569 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246243 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee4f80f2-7676-489e-9fe0-99c52e384790-cnibin\") pod \"multus-additional-cni-plugins-k9th4\" (UID: \"ee4f80f2-7676-489e-9fe0-99c52e384790\") " pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.247569 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246268 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb9bn\" (UniqueName: \"kubernetes.io/projected/9fefcefc-4128-4422-98de-b17e5949cfc7-kube-api-access-mb9bn\") pod \"network-metrics-daemon-b4gcd\" (UID: \"9fefcefc-4128-4422-98de-b17e5949cfc7\") " pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:06:52.247569 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246321 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2nmj\" (UniqueName: \"kubernetes.io/projected/0a7d54c6-9262-4462-8b4b-f35a678e2ddf-kube-api-access-m2nmj\") pod \"node-ca-dgz8s\" (UID: \"0a7d54c6-9262-4462-8b4b-f35a678e2ddf\") " pod="openshift-image-registry/node-ca-dgz8s" Jul 09 04:06:52.247569 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246355 2534 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 04:06:52.247569 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246421 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7867a42f-0f05-4c3c-8e48-fa68962734c1-iptables-alerter-script\") pod \"iptables-alerter-rh79z\" (UID: \"7867a42f-0f05-4c3c-8e48-fa68962734c1\") " pod="openshift-network-operator/iptables-alerter-rh79z" Jul 09 04:06:52.247569 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246445 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-host-slash\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.247569 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246467 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-etc-kubernetes\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.247569 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246492 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2kww\" (UniqueName: \"kubernetes.io/projected/ee4f80f2-7676-489e-9fe0-99c52e384790-kube-api-access-j2kww\") pod \"multus-additional-cni-plugins-k9th4\" (UID: \"ee4f80f2-7676-489e-9fe0-99c52e384790\") " pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.248469 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246516 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9d4cab2f-0478-4c53-9966-d96d5e18f2dd-socket-dir\") pod \"aws-ebs-csi-driver-node-58mgf\" (UID: \"9d4cab2f-0478-4c53-9966-d96d5e18f2dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" Jul 09 04:06:52.248469 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246540 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84mm7\" (UniqueName: \"kubernetes.io/projected/9d4cab2f-0478-4c53-9966-d96d5e18f2dd-kube-api-access-84mm7\") pod \"aws-ebs-csi-driver-node-58mgf\" (UID: \"9d4cab2f-0478-4c53-9966-d96d5e18f2dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" Jul 09 04:06:52.248469 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246555 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0a7d54c6-9262-4462-8b4b-f35a678e2ddf-serviceca\") pod \"node-ca-dgz8s\" (UID: \"0a7d54c6-9262-4462-8b4b-f35a678e2ddf\") " pod="openshift-image-registry/node-ca-dgz8s" Jul 09 04:06:52.248469 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246566 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-dbus\") pod \"global-pull-secret-syncer-krxxf\" (UID: \"2c0aaa25-8a07-4da9-8d7f-b8abd6026f36\") " pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:06:52.248469 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246593 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-etc-sysctl-d\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.248469 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246616 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee4f80f2-7676-489e-9fe0-99c52e384790-cni-binary-copy\") pod \"multus-additional-cni-plugins-k9th4\" (UID: \"ee4f80f2-7676-489e-9fe0-99c52e384790\") " pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.248469 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246641 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ee4f80f2-7676-489e-9fe0-99c52e384790-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k9th4\" (UID: \"ee4f80f2-7676-489e-9fe0-99c52e384790\") " pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.248469 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246669 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/758a58c5-cb89-447a-b25a-6c3d87b1ba11-etc-tuned\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.248469 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246691 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-kubelet-config\") pod \"global-pull-secret-syncer-krxxf\" (UID: \"2c0aaa25-8a07-4da9-8d7f-b8abd6026f36\") " pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:06:52.248469 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246719 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sx6p\" (UniqueName: \"kubernetes.io/projected/7867a42f-0f05-4c3c-8e48-fa68962734c1-kube-api-access-8sx6p\") pod \"iptables-alerter-rh79z\" (UID: \"7867a42f-0f05-4c3c-8e48-fa68962734c1\") " pod="openshift-network-operator/iptables-alerter-rh79z" Jul 09 04:06:52.248469 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246829 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.248469 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246855 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-os-release\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.248469 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246881 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-host-run-k8s-cni-cncf-io\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.248469 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246910 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmpnc\" (UniqueName: \"kubernetes.io/projected/8104c5b9-011c-4e10-9a15-53bdd52870f0-kube-api-access-hmpnc\") pod \"node-resolver-zpr9q\" (UID: \"8104c5b9-011c-4e10-9a15-53bdd52870f0\") " pod="openshift-dns/node-resolver-zpr9q" Jul 09 04:06:52.248469 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246934 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-etc-sysconfig\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.248469 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246973 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5wq8\" (UniqueName: \"kubernetes.io/projected/758a58c5-cb89-447a-b25a-6c3d87b1ba11-kube-api-access-p5wq8\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.249319 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.246997 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-host-run-ovn-kubernetes\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.249319 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.247021 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-dbus\") pod \"global-pull-secret-syncer-krxxf\" (UID: \"2c0aaa25-8a07-4da9-8d7f-b8abd6026f36\") " pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:06:52.249319 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.247023 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-host-cni-netd\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.249319 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.247059 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-multus-cni-dir\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.249319 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.247094 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-host-var-lib-cni-bin\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.249319 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.247127 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-hostroot\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.249319 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.247157 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-etc-sysctl-d\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.249319 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.247156 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-host-run-multus-certs\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.249319 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.247127 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-kubelet-config\") pod \"global-pull-secret-syncer-krxxf\" (UID: \"2c0aaa25-8a07-4da9-8d7f-b8abd6026f36\") " pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:06:52.249319 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.247203 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-var-lib-kubelet\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.249319 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.247231 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxdh5\" (UniqueName: \"kubernetes.io/projected/8e433de1-ad41-4547-b130-e06490dba1bb-kube-api-access-dxdh5\") pod \"network-check-target-8xpfm\" (UID: \"8e433de1-ad41-4547-b130-e06490dba1bb\") " pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:06:52.249319 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.247257 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4b157285-2103-4dc1-b461-ac80ec29eb97-ovnkube-script-lib\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.249319 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.247279 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-cnibin\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.249319 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.247303 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drxqs\" (UniqueName: \"kubernetes.io/projected/889db0de-781d-43d6-9df1-7a9eb15a1794-kube-api-access-drxqs\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.249319 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.247559 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-var-lib-kubelet\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.249319 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.247573 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/758a58c5-cb89-447a-b25a-6c3d87b1ba11-etc-sysconfig\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.250130 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.250038 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-153.ec2.internal" event={"ID":"67b85ef8254593a9d9a03d6a7f9822a9","Type":"ContainerStarted","Data":"863ba7342a980a1197b393904943298e94aa2bbae36bc2260048e389e98746c3"} Jul 09 04:06:52.251150 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.251126 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/758a58c5-cb89-447a-b25a-6c3d87b1ba11-etc-tuned\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.251798 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.251773 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-153.ec2.internal" event={"ID":"e28bfc992df73a71242390a6bf756269","Type":"ContainerStarted","Data":"eb880e83114428ffa6caa0898fc537a39a33d9aaa1f3d84740316e21ceee2a4c"} Jul 09 04:06:52.253238 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.252306 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/758a58c5-cb89-447a-b25a-6c3d87b1ba11-tmp\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.257195 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.257151 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2nmj\" (UniqueName: \"kubernetes.io/projected/0a7d54c6-9262-4462-8b4b-f35a678e2ddf-kube-api-access-m2nmj\") pod \"node-ca-dgz8s\" (UID: \"0a7d54c6-9262-4462-8b4b-f35a678e2ddf\") " pod="openshift-image-registry/node-ca-dgz8s" Jul 09 04:06:52.257649 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.257582 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmpnc\" (UniqueName: \"kubernetes.io/projected/8104c5b9-011c-4e10-9a15-53bdd52870f0-kube-api-access-hmpnc\") pod \"node-resolver-zpr9q\" (UID: \"8104c5b9-011c-4e10-9a15-53bdd52870f0\") " pod="openshift-dns/node-resolver-zpr9q" Jul 09 04:06:52.259200 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.259180 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5wq8\" (UniqueName: \"kubernetes.io/projected/758a58c5-cb89-447a-b25a-6c3d87b1ba11-kube-api-access-p5wq8\") pod \"tuned-ks45b\" (UID: \"758a58c5-cb89-447a-b25a-6c3d87b1ba11\") " pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.348591 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.348463 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4b157285-2103-4dc1-b461-ac80ec29eb97-ovnkube-script-lib\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.348591 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.348517 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-cnibin\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.348591 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.348551 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drxqs\" (UniqueName: \"kubernetes.io/projected/889db0de-781d-43d6-9df1-7a9eb15a1794-kube-api-access-drxqs\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.348591 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.348583 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7867a42f-0f05-4c3c-8e48-fa68962734c1-host-slash\") pod \"iptables-alerter-rh79z\" (UID: \"7867a42f-0f05-4c3c-8e48-fa68962734c1\") " pod="openshift-network-operator/iptables-alerter-rh79z" Jul 09 04:06:52.348873 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.348609 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-run-systemd\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.348873 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.348634 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-run-ovn\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.348873 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.348657 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4b157285-2103-4dc1-b461-ac80ec29eb97-ovn-node-metrics-cert\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.348873 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.348681 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee4f80f2-7676-489e-9fe0-99c52e384790-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k9th4\" (UID: \"ee4f80f2-7676-489e-9fe0-99c52e384790\") " pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.348873 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.348706 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9d4cab2f-0478-4c53-9966-d96d5e18f2dd-sys-fs\") pod \"aws-ebs-csi-driver-node-58mgf\" (UID: \"9d4cab2f-0478-4c53-9966-d96d5e18f2dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" Jul 09 04:06:52.348873 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.348761 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-systemd-units\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.348873 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.348787 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-etc-openvswitch\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.348873 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.348810 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-node-log\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.348873 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.348835 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/889db0de-781d-43d6-9df1-7a9eb15a1794-cni-binary-copy\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.348873 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.348859 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9d4cab2f-0478-4c53-9966-d96d5e18f2dd-etc-selinux\") pod \"aws-ebs-csi-driver-node-58mgf\" (UID: \"9d4cab2f-0478-4c53-9966-d96d5e18f2dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" Jul 09 04:06:52.349200 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.348887 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-host-run-netns\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.349200 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.348911 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-host-cni-bin\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.349200 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.348935 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-multus-socket-dir-parent\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.349200 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.348960 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-log-socket\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.349200 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.348985 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee4f80f2-7676-489e-9fe0-99c52e384790-system-cni-dir\") pod \"multus-additional-cni-plugins-k9th4\" (UID: \"ee4f80f2-7676-489e-9fe0-99c52e384790\") " pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.349200 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349009 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e6cff921-2b0d-4748-ab0f-642fc9ff3d7e-agent-certs\") pod \"konnectivity-agent-xksp2\" (UID: \"e6cff921-2b0d-4748-ab0f-642fc9ff3d7e\") " pod="kube-system/konnectivity-agent-xksp2" Jul 09 04:06:52.349200 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349034 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9d4cab2f-0478-4c53-9966-d96d5e18f2dd-registration-dir\") pod \"aws-ebs-csi-driver-node-58mgf\" (UID: \"9d4cab2f-0478-4c53-9966-d96d5e18f2dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" Jul 09 04:06:52.349200 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349059 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-run-openvswitch\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.349200 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349062 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4b157285-2103-4dc1-b461-ac80ec29eb97-ovnkube-script-lib\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.349200 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349084 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-host-var-lib-kubelet\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.349200 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349110 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ee4f80f2-7676-489e-9fe0-99c52e384790-os-release\") pod \"multus-additional-cni-plugins-k9th4\" (UID: \"ee4f80f2-7676-489e-9fe0-99c52e384790\") " pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.349200 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349134 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs\") pod \"network-metrics-daemon-b4gcd\" (UID: \"9fefcefc-4128-4422-98de-b17e5949cfc7\") " pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:06:52.349200 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349183 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-host-kubelet\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.349754 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349210 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-system-cni-dir\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.349754 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349242 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ee4f80f2-7676-489e-9fe0-99c52e384790-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k9th4\" (UID: \"ee4f80f2-7676-489e-9fe0-99c52e384790\") " pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.349754 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349271 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4b157285-2103-4dc1-b461-ac80ec29eb97-env-overrides\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.349754 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349298 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-host-run-netns\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.349754 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349322 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-host-var-lib-cni-multus\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.349754 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349345 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-multus-conf-dir\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.349754 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349404 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/889db0de-781d-43d6-9df1-7a9eb15a1794-multus-daemon-config\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.349754 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349432 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e6cff921-2b0d-4748-ab0f-642fc9ff3d7e-konnectivity-ca\") pod \"konnectivity-agent-xksp2\" (UID: \"e6cff921-2b0d-4748-ab0f-642fc9ff3d7e\") " pod="kube-system/konnectivity-agent-xksp2" Jul 09 04:06:52.349754 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349462 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-multus-socket-dir-parent\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.349754 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349468 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-var-lib-openvswitch\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.349754 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349491 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9d4cab2f-0478-4c53-9966-d96d5e18f2dd-sys-fs\") pod \"aws-ebs-csi-driver-node-58mgf\" (UID: \"9d4cab2f-0478-4c53-9966-d96d5e18f2dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" Jul 09 04:06:52.349754 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349520 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-var-lib-openvswitch\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.349754 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349536 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-systemd-units\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.349754 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349572 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-log-socket\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.349754 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349584 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-cnibin\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.349754 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349625 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee4f80f2-7676-489e-9fe0-99c52e384790-system-cni-dir\") pod \"multus-additional-cni-plugins-k9th4\" (UID: \"ee4f80f2-7676-489e-9fe0-99c52e384790\") " pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.349754 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349622 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-run-ovn\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.349754 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349644 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9d4cab2f-0478-4c53-9966-d96d5e18f2dd-registration-dir\") pod \"aws-ebs-csi-driver-node-58mgf\" (UID: \"9d4cab2f-0478-4c53-9966-d96d5e18f2dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" Jul 09 04:06:52.350586 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349678 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-etc-openvswitch\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.350586 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349687 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-run-openvswitch\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.350586 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349731 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-node-log\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.350586 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349742 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-host-var-lib-kubelet\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.350586 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.349816 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ee4f80f2-7676-489e-9fe0-99c52e384790-os-release\") pod \"multus-additional-cni-plugins-k9th4\" (UID: \"ee4f80f2-7676-489e-9fe0-99c52e384790\") " pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.350586 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:52.349910 2534 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:06:52.350586 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:52.349967 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs podName:9fefcefc-4128-4422-98de-b17e5949cfc7 nodeName:}" failed. No retries permitted until 2026-07-09 04:06:52.849949548 +0000 UTC m=+3.073568029 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs") pod "network-metrics-daemon-b4gcd" (UID: "9fefcefc-4128-4422-98de-b17e5949cfc7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:06:52.350586 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350188 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-host-kubelet\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.350586 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350236 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/889db0de-781d-43d6-9df1-7a9eb15a1794-cni-binary-copy\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.350586 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350257 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-system-cni-dir\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.350586 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350290 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-host-run-netns\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.350586 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350395 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9d4cab2f-0478-4c53-9966-d96d5e18f2dd-etc-selinux\") pod \"aws-ebs-csi-driver-node-58mgf\" (UID: \"9d4cab2f-0478-4c53-9966-d96d5e18f2dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" Jul 09 04:06:52.350586 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350442 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-host-cni-bin\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.350586 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350472 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4b157285-2103-4dc1-b461-ac80ec29eb97-ovnkube-config\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.350586 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350504 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56z2j\" (UniqueName: \"kubernetes.io/projected/4b157285-2103-4dc1-b461-ac80ec29eb97-kube-api-access-56z2j\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.350586 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350531 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9d4cab2f-0478-4c53-9966-d96d5e18f2dd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-58mgf\" (UID: \"9d4cab2f-0478-4c53-9966-d96d5e18f2dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" Jul 09 04:06:52.350586 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350555 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9d4cab2f-0478-4c53-9966-d96d5e18f2dd-device-dir\") pod \"aws-ebs-csi-driver-node-58mgf\" (UID: \"9d4cab2f-0478-4c53-9966-d96d5e18f2dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" Jul 09 04:06:52.351245 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350580 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee4f80f2-7676-489e-9fe0-99c52e384790-cnibin\") pod \"multus-additional-cni-plugins-k9th4\" (UID: \"ee4f80f2-7676-489e-9fe0-99c52e384790\") " pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.351245 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350605 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb9bn\" (UniqueName: \"kubernetes.io/projected/9fefcefc-4128-4422-98de-b17e5949cfc7-kube-api-access-mb9bn\") pod \"network-metrics-daemon-b4gcd\" (UID: \"9fefcefc-4128-4422-98de-b17e5949cfc7\") " pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:06:52.351245 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350628 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7867a42f-0f05-4c3c-8e48-fa68962734c1-iptables-alerter-script\") pod \"iptables-alerter-rh79z\" (UID: \"7867a42f-0f05-4c3c-8e48-fa68962734c1\") " pod="openshift-network-operator/iptables-alerter-rh79z" Jul 09 04:06:52.351245 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350644 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-host-slash\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.351245 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350660 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-etc-kubernetes\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.351245 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350667 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee4f80f2-7676-489e-9fe0-99c52e384790-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k9th4\" (UID: \"ee4f80f2-7676-489e-9fe0-99c52e384790\") " pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.351245 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350675 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2kww\" (UniqueName: \"kubernetes.io/projected/ee4f80f2-7676-489e-9fe0-99c52e384790-kube-api-access-j2kww\") pod \"multus-additional-cni-plugins-k9th4\" (UID: \"ee4f80f2-7676-489e-9fe0-99c52e384790\") " pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.351245 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350732 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9d4cab2f-0478-4c53-9966-d96d5e18f2dd-socket-dir\") pod \"aws-ebs-csi-driver-node-58mgf\" (UID: \"9d4cab2f-0478-4c53-9966-d96d5e18f2dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" Jul 09 04:06:52.351245 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350737 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ee4f80f2-7676-489e-9fe0-99c52e384790-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k9th4\" (UID: \"ee4f80f2-7676-489e-9fe0-99c52e384790\") " pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.351245 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350762 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84mm7\" (UniqueName: \"kubernetes.io/projected/9d4cab2f-0478-4c53-9966-d96d5e18f2dd-kube-api-access-84mm7\") pod \"aws-ebs-csi-driver-node-58mgf\" (UID: \"9d4cab2f-0478-4c53-9966-d96d5e18f2dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" Jul 09 04:06:52.351245 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350795 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee4f80f2-7676-489e-9fe0-99c52e384790-cni-binary-copy\") pod \"multus-additional-cni-plugins-k9th4\" (UID: \"ee4f80f2-7676-489e-9fe0-99c52e384790\") " pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.351245 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350823 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ee4f80f2-7676-489e-9fe0-99c52e384790-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k9th4\" (UID: \"ee4f80f2-7676-489e-9fe0-99c52e384790\") " pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.351245 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350856 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sx6p\" (UniqueName: \"kubernetes.io/projected/7867a42f-0f05-4c3c-8e48-fa68962734c1-kube-api-access-8sx6p\") pod \"iptables-alerter-rh79z\" (UID: \"7867a42f-0f05-4c3c-8e48-fa68962734c1\") " pod="openshift-network-operator/iptables-alerter-rh79z" Jul 09 04:06:52.351245 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350882 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.351245 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350911 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-os-release\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.351245 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350937 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-host-run-k8s-cni-cncf-io\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.352068 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350968 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-host-run-ovn-kubernetes\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.352068 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.350991 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-host-cni-netd\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.352068 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.351006 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4b157285-2103-4dc1-b461-ac80ec29eb97-env-overrides\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.352068 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.351014 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-multus-cni-dir\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.352068 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.351041 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-host-var-lib-cni-bin\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.352068 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.351052 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-host-var-lib-cni-multus\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.352068 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.351066 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-hostroot\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.352068 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.351081 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-multus-conf-dir\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.352068 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.351093 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-host-run-multus-certs\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.352068 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.351122 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxdh5\" (UniqueName: \"kubernetes.io/projected/8e433de1-ad41-4547-b130-e06490dba1bb-kube-api-access-dxdh5\") pod \"network-check-target-8xpfm\" (UID: \"8e433de1-ad41-4547-b130-e06490dba1bb\") " pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:06:52.352068 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.351322 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4b157285-2103-4dc1-b461-ac80ec29eb97-ovnkube-config\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.352068 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.351514 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9d4cab2f-0478-4c53-9966-d96d5e18f2dd-socket-dir\") pod \"aws-ebs-csi-driver-node-58mgf\" (UID: \"9d4cab2f-0478-4c53-9966-d96d5e18f2dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" Jul 09 04:06:52.352068 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.351606 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9d4cab2f-0478-4c53-9966-d96d5e18f2dd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-58mgf\" (UID: \"9d4cab2f-0478-4c53-9966-d96d5e18f2dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" Jul 09 04:06:52.352068 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.351660 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e6cff921-2b0d-4748-ab0f-642fc9ff3d7e-konnectivity-ca\") pod \"konnectivity-agent-xksp2\" (UID: \"e6cff921-2b0d-4748-ab0f-642fc9ff3d7e\") " pod="kube-system/konnectivity-agent-xksp2" Jul 09 04:06:52.352068 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.351662 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9d4cab2f-0478-4c53-9966-d96d5e18f2dd-device-dir\") pod \"aws-ebs-csi-driver-node-58mgf\" (UID: \"9d4cab2f-0478-4c53-9966-d96d5e18f2dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" Jul 09 04:06:52.352068 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.351697 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee4f80f2-7676-489e-9fe0-99c52e384790-cnibin\") pod \"multus-additional-cni-plugins-k9th4\" (UID: \"ee4f80f2-7676-489e-9fe0-99c52e384790\") " pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.352068 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.351751 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-host-slash\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.352068 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.351790 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-etc-kubernetes\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.352979 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.351832 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-host-cni-netd\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.352979 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.351901 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-run-systemd\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.352979 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.351969 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-os-release\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.352979 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.351969 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-host-var-lib-cni-bin\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.352979 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.352011 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.352979 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.352031 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-multus-cni-dir\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.352979 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.352049 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-host-run-netns\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.352979 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.352075 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-host-run-k8s-cni-cncf-io\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.352979 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.352147 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b157285-2103-4dc1-b461-ac80ec29eb97-host-run-ovn-kubernetes\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.352979 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.352183 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-hostroot\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.352979 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.352481 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7867a42f-0f05-4c3c-8e48-fa68962734c1-host-slash\") pod \"iptables-alerter-rh79z\" (UID: \"7867a42f-0f05-4c3c-8e48-fa68962734c1\") " pod="openshift-network-operator/iptables-alerter-rh79z" Jul 09 04:06:52.352979 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.352551 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/889db0de-781d-43d6-9df1-7a9eb15a1794-host-run-multus-certs\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.352979 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.352707 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/889db0de-781d-43d6-9df1-7a9eb15a1794-multus-daemon-config\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.352979 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.352820 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4b157285-2103-4dc1-b461-ac80ec29eb97-ovn-node-metrics-cert\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.353730 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.352998 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7867a42f-0f05-4c3c-8e48-fa68962734c1-iptables-alerter-script\") pod \"iptables-alerter-rh79z\" (UID: \"7867a42f-0f05-4c3c-8e48-fa68962734c1\") " pod="openshift-network-operator/iptables-alerter-rh79z" Jul 09 04:06:52.353730 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.353479 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee4f80f2-7676-489e-9fe0-99c52e384790-cni-binary-copy\") pod \"multus-additional-cni-plugins-k9th4\" (UID: \"ee4f80f2-7676-489e-9fe0-99c52e384790\") " pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.353730 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.353554 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ee4f80f2-7676-489e-9fe0-99c52e384790-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k9th4\" (UID: \"ee4f80f2-7676-489e-9fe0-99c52e384790\") " pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.354648 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.354629 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e6cff921-2b0d-4748-ab0f-642fc9ff3d7e-agent-certs\") pod \"konnectivity-agent-xksp2\" (UID: \"e6cff921-2b0d-4748-ab0f-642fc9ff3d7e\") " pod="kube-system/konnectivity-agent-xksp2" Jul 09 04:06:52.359909 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:52.359889 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 04:06:52.360013 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:52.359915 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 04:06:52.360013 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:52.359929 2534 projected.go:196] Error preparing data for projected volume kube-api-access-dxdh5 for pod openshift-network-diagnostics/network-check-target-8xpfm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:06:52.360013 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:52.359984 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e433de1-ad41-4547-b130-e06490dba1bb-kube-api-access-dxdh5 podName:8e433de1-ad41-4547-b130-e06490dba1bb nodeName:}" failed. No retries permitted until 2026-07-09 04:06:52.859966506 +0000 UTC m=+3.083584989 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dxdh5" (UniqueName: "kubernetes.io/projected/8e433de1-ad41-4547-b130-e06490dba1bb-kube-api-access-dxdh5") pod "network-check-target-8xpfm" (UID: "8e433de1-ad41-4547-b130-e06490dba1bb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:06:52.362011 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.361983 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drxqs\" (UniqueName: \"kubernetes.io/projected/889db0de-781d-43d6-9df1-7a9eb15a1794-kube-api-access-drxqs\") pod \"multus-bv4mn\" (UID: \"889db0de-781d-43d6-9df1-7a9eb15a1794\") " pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.363242 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.363218 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2kww\" (UniqueName: \"kubernetes.io/projected/ee4f80f2-7676-489e-9fe0-99c52e384790-kube-api-access-j2kww\") pod \"multus-additional-cni-plugins-k9th4\" (UID: \"ee4f80f2-7676-489e-9fe0-99c52e384790\") " pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.364338 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.364315 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84mm7\" (UniqueName: \"kubernetes.io/projected/9d4cab2f-0478-4c53-9966-d96d5e18f2dd-kube-api-access-84mm7\") pod \"aws-ebs-csi-driver-node-58mgf\" (UID: \"9d4cab2f-0478-4c53-9966-d96d5e18f2dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" Jul 09 04:06:52.365672 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.365649 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb9bn\" (UniqueName: \"kubernetes.io/projected/9fefcefc-4128-4422-98de-b17e5949cfc7-kube-api-access-mb9bn\") pod \"network-metrics-daemon-b4gcd\" (UID: \"9fefcefc-4128-4422-98de-b17e5949cfc7\") " pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:06:52.367599 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.367574 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sx6p\" (UniqueName: \"kubernetes.io/projected/7867a42f-0f05-4c3c-8e48-fa68962734c1-kube-api-access-8sx6p\") pod \"iptables-alerter-rh79z\" (UID: \"7867a42f-0f05-4c3c-8e48-fa68962734c1\") " pod="openshift-network-operator/iptables-alerter-rh79z" Jul 09 04:06:52.367732 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.367704 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56z2j\" (UniqueName: \"kubernetes.io/projected/4b157285-2103-4dc1-b461-ac80ec29eb97-kube-api-access-56z2j\") pod \"ovnkube-node-pkb8q\" (UID: \"4b157285-2103-4dc1-b461-ac80ec29eb97\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.441455 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.441416 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zpr9q" Jul 09 04:06:52.453247 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.453222 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ks45b" Jul 09 04:06:52.454690 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:52.454663 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8104c5b9_011c_4e10_9a15_53bdd52870f0.slice/crio-1cd66e93956cc387b3d800b693f31a1872b908de1f9345d1458b9a543c679b74 WatchSource:0}: Error finding container 1cd66e93956cc387b3d800b693f31a1872b908de1f9345d1458b9a543c679b74: Status 404 returned error can't find the container with id 1cd66e93956cc387b3d800b693f31a1872b908de1f9345d1458b9a543c679b74 Jul 09 04:06:52.462217 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.462197 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dgz8s" Jul 09 04:06:52.463759 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:52.463730 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod758a58c5_cb89_447a_b25a_6c3d87b1ba11.slice/crio-145964d302217ba624997f7018e54dbcf059354db64091aa6859b40c5698a592 WatchSource:0}: Error finding container 145964d302217ba624997f7018e54dbcf059354db64091aa6859b40c5698a592: Status 404 returned error can't find the container with id 145964d302217ba624997f7018e54dbcf059354db64091aa6859b40c5698a592 Jul 09 04:06:52.472022 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.471988 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:06:52.473713 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:52.473683 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a7d54c6_9262_4462_8b4b_f35a678e2ddf.slice/crio-d993e9a5a0b261cf6dd2ce0db00dd403587c1df5ec79abf504eed3048a3ffc85 WatchSource:0}: Error finding container d993e9a5a0b261cf6dd2ce0db00dd403587c1df5ec79abf504eed3048a3ffc85: Status 404 returned error can't find the container with id d993e9a5a0b261cf6dd2ce0db00dd403587c1df5ec79abf504eed3048a3ffc85 Jul 09 04:06:52.479783 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.479760 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k9th4" Jul 09 04:06:52.484637 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:52.484612 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b157285_2103_4dc1_b461_ac80ec29eb97.slice/crio-9300b6f3fedadd97329e92f2dd2e175f9dc0f64648f54dea58e7491cb1a73e34 WatchSource:0}: Error finding container 9300b6f3fedadd97329e92f2dd2e175f9dc0f64648f54dea58e7491cb1a73e34: Status 404 returned error can't find the container with id 9300b6f3fedadd97329e92f2dd2e175f9dc0f64648f54dea58e7491cb1a73e34 Jul 09 04:06:52.487433 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.487411 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xksp2" Jul 09 04:06:52.489582 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:52.489562 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee4f80f2_7676_489e_9fe0_99c52e384790.slice/crio-267fe2f65c9be0209d8d4c7a53bb14330df9191748495ed1a0ad34dbaf35ca0c WatchSource:0}: Error finding container 267fe2f65c9be0209d8d4c7a53bb14330df9191748495ed1a0ad34dbaf35ca0c: Status 404 returned error can't find the container with id 267fe2f65c9be0209d8d4c7a53bb14330df9191748495ed1a0ad34dbaf35ca0c Jul 09 04:06:52.495369 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.495345 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" Jul 09 04:06:52.497253 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:52.497233 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6cff921_2b0d_4748_ab0f_642fc9ff3d7e.slice/crio-669e3647f04a8c10019640a3e58cecf78a3109bec00b22089ea71a38ef7ce883 WatchSource:0}: Error finding container 669e3647f04a8c10019640a3e58cecf78a3109bec00b22089ea71a38ef7ce883: Status 404 returned error can't find the container with id 669e3647f04a8c10019640a3e58cecf78a3109bec00b22089ea71a38ef7ce883 Jul 09 04:06:52.505871 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.505848 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bv4mn" Jul 09 04:06:52.513047 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.513020 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rh79z" Jul 09 04:06:52.753938 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.753837 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-original-pull-secret\") pod \"global-pull-secret-syncer-krxxf\" (UID: \"2c0aaa25-8a07-4da9-8d7f-b8abd6026f36\") " pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:06:52.754104 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:52.754017 2534 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 04:06:52.754104 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:52.754096 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-original-pull-secret podName:2c0aaa25-8a07-4da9-8d7f-b8abd6026f36 nodeName:}" failed. No retries permitted until 2026-07-09 04:06:53.754076376 +0000 UTC m=+3.977694844 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-original-pull-secret") pod "global-pull-secret-syncer-krxxf" (UID: "2c0aaa25-8a07-4da9-8d7f-b8abd6026f36") : object "kube-system"/"original-pull-secret" not registered Jul 09 04:06:52.855069 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.855039 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs\") pod \"network-metrics-daemon-b4gcd\" (UID: \"9fefcefc-4128-4422-98de-b17e5949cfc7\") " pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:06:52.855210 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:52.855162 2534 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:06:52.855260 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:52.855219 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs podName:9fefcefc-4128-4422-98de-b17e5949cfc7 nodeName:}" failed. No retries permitted until 2026-07-09 04:06:53.855204311 +0000 UTC m=+4.078822778 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs") pod "network-metrics-daemon-b4gcd" (UID: "9fefcefc-4128-4422-98de-b17e5949cfc7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:06:52.955763 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:52.955720 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxdh5\" (UniqueName: \"kubernetes.io/projected/8e433de1-ad41-4547-b130-e06490dba1bb-kube-api-access-dxdh5\") pod \"network-check-target-8xpfm\" (UID: \"8e433de1-ad41-4547-b130-e06490dba1bb\") " pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:06:52.955932 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:52.955899 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 04:06:52.955932 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:52.955922 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 04:06:52.955932 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:52.955933 2534 projected.go:196] Error preparing data for projected volume kube-api-access-dxdh5 for pod openshift-network-diagnostics/network-check-target-8xpfm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:06:52.956045 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:52.956007 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e433de1-ad41-4547-b130-e06490dba1bb-kube-api-access-dxdh5 podName:8e433de1-ad41-4547-b130-e06490dba1bb nodeName:}" failed. No retries permitted until 2026-07-09 04:06:53.955986595 +0000 UTC m=+4.179605063 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dxdh5" (UniqueName: "kubernetes.io/projected/8e433de1-ad41-4547-b130-e06490dba1bb-kube-api-access-dxdh5") pod "network-check-target-8xpfm" (UID: "8e433de1-ad41-4547-b130-e06490dba1bb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:06:53.178201 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:53.178118 2534 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 04:01:53 +0000 UTC" deadline="2028-02-10 10:11:28.047774697 +0000 UTC" Jul 09 04:06:53.178201 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:53.178153 2534 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13950h4m34.869626615s" Jul 09 04:06:53.244832 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:53.244799 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:06:53.245019 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:53.244952 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4gcd" podUID="9fefcefc-4128-4422-98de-b17e5949cfc7" Jul 09 04:06:53.245097 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:53.245020 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:06:53.245166 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:53.245142 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xpfm" podUID="8e433de1-ad41-4547-b130-e06490dba1bb" Jul 09 04:06:53.254991 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:53.254959 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k9th4" event={"ID":"ee4f80f2-7676-489e-9fe0-99c52e384790","Type":"ContainerStarted","Data":"267fe2f65c9be0209d8d4c7a53bb14330df9191748495ed1a0ad34dbaf35ca0c"} Jul 09 04:06:53.256206 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:53.256179 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" event={"ID":"4b157285-2103-4dc1-b461-ac80ec29eb97","Type":"ContainerStarted","Data":"9300b6f3fedadd97329e92f2dd2e175f9dc0f64648f54dea58e7491cb1a73e34"} Jul 09 04:06:53.257506 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:53.257452 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dgz8s" event={"ID":"0a7d54c6-9262-4462-8b4b-f35a678e2ddf","Type":"ContainerStarted","Data":"d993e9a5a0b261cf6dd2ce0db00dd403587c1df5ec79abf504eed3048a3ffc85"} Jul 09 04:06:53.258957 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:53.258925 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ks45b" event={"ID":"758a58c5-cb89-447a-b25a-6c3d87b1ba11","Type":"ContainerStarted","Data":"145964d302217ba624997f7018e54dbcf059354db64091aa6859b40c5698a592"} Jul 09 04:06:53.260182 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:53.260158 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zpr9q" event={"ID":"8104c5b9-011c-4e10-9a15-53bdd52870f0","Type":"ContainerStarted","Data":"1cd66e93956cc387b3d800b693f31a1872b908de1f9345d1458b9a543c679b74"} Jul 09 04:06:53.261489 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:53.261464 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xksp2" event={"ID":"e6cff921-2b0d-4748-ab0f-642fc9ff3d7e","Type":"ContainerStarted","Data":"669e3647f04a8c10019640a3e58cecf78a3109bec00b22089ea71a38ef7ce883"} Jul 09 04:06:53.614395 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:53.614343 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7867a42f_0f05_4c3c_8e48_fa68962734c1.slice/crio-665a1d062680da27f82d8d8fca8a339ad900399f191df655c4450622545a1826 WatchSource:0}: Error finding container 665a1d062680da27f82d8d8fca8a339ad900399f191df655c4450622545a1826: Status 404 returned error can't find the container with id 665a1d062680da27f82d8d8fca8a339ad900399f191df655c4450622545a1826 Jul 09 04:06:53.620020 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:53.619884 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d4cab2f_0478_4c53_9966_d96d5e18f2dd.slice/crio-1ce95a56b149dd15f614640bf0f5607f033ef5ae34bcaa55b3925f210d0278e4 WatchSource:0}: Error finding container 1ce95a56b149dd15f614640bf0f5607f033ef5ae34bcaa55b3925f210d0278e4: Status 404 returned error can't find the container with id 1ce95a56b149dd15f614640bf0f5607f033ef5ae34bcaa55b3925f210d0278e4 Jul 09 04:06:53.621944 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:06:53.621773 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod889db0de_781d_43d6_9df1_7a9eb15a1794.slice/crio-0cea159a8261a7cb57a82178727c37939f9c4f7fd5b7a74c9240fe9d72bf9525 WatchSource:0}: Error finding container 0cea159a8261a7cb57a82178727c37939f9c4f7fd5b7a74c9240fe9d72bf9525: Status 404 returned error can't find the container with id 0cea159a8261a7cb57a82178727c37939f9c4f7fd5b7a74c9240fe9d72bf9525 Jul 09 04:06:53.761682 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:53.761423 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-original-pull-secret\") pod \"global-pull-secret-syncer-krxxf\" (UID: \"2c0aaa25-8a07-4da9-8d7f-b8abd6026f36\") " pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:06:53.761916 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:53.761738 2534 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 04:06:53.761916 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:53.761817 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-original-pull-secret podName:2c0aaa25-8a07-4da9-8d7f-b8abd6026f36 nodeName:}" failed. No retries permitted until 2026-07-09 04:06:55.761796495 +0000 UTC m=+5.985414963 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-original-pull-secret") pod "global-pull-secret-syncer-krxxf" (UID: "2c0aaa25-8a07-4da9-8d7f-b8abd6026f36") : object "kube-system"/"original-pull-secret" not registered Jul 09 04:06:53.863556 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:53.862847 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs\") pod \"network-metrics-daemon-b4gcd\" (UID: \"9fefcefc-4128-4422-98de-b17e5949cfc7\") " pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:06:53.863556 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:53.863032 2534 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:06:53.863556 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:53.863099 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs podName:9fefcefc-4128-4422-98de-b17e5949cfc7 nodeName:}" failed. No retries permitted until 2026-07-09 04:06:55.863079914 +0000 UTC m=+6.086698385 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs") pod "network-metrics-daemon-b4gcd" (UID: "9fefcefc-4128-4422-98de-b17e5949cfc7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:06:53.963418 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:53.963377 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxdh5\" (UniqueName: \"kubernetes.io/projected/8e433de1-ad41-4547-b130-e06490dba1bb-kube-api-access-dxdh5\") pod \"network-check-target-8xpfm\" (UID: \"8e433de1-ad41-4547-b130-e06490dba1bb\") " pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:06:53.963589 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:53.963544 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 04:06:53.963589 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:53.963564 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 04:06:53.963589 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:53.963576 2534 projected.go:196] Error preparing data for projected volume kube-api-access-dxdh5 for pod openshift-network-diagnostics/network-check-target-8xpfm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:06:53.963764 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:53.963637 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e433de1-ad41-4547-b130-e06490dba1bb-kube-api-access-dxdh5 podName:8e433de1-ad41-4547-b130-e06490dba1bb nodeName:}" failed. No retries permitted until 2026-07-09 04:06:55.963617031 +0000 UTC m=+6.187235522 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dxdh5" (UniqueName: "kubernetes.io/projected/8e433de1-ad41-4547-b130-e06490dba1bb-kube-api-access-dxdh5") pod "network-check-target-8xpfm" (UID: "8e433de1-ad41-4547-b130-e06490dba1bb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:06:54.180437 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:54.180332 2534 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 04:01:53 +0000 UTC" deadline="2027-12-29 08:46:35.246690025 +0000 UTC" Jul 09 04:06:54.180437 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:54.180400 2534 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12916h39m41.066294885s" Jul 09 04:06:54.247213 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:54.246690 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:06:54.247213 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:54.246836 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krxxf" podUID="2c0aaa25-8a07-4da9-8d7f-b8abd6026f36" Jul 09 04:06:54.267710 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:54.267674 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rh79z" event={"ID":"7867a42f-0f05-4c3c-8e48-fa68962734c1","Type":"ContainerStarted","Data":"665a1d062680da27f82d8d8fca8a339ad900399f191df655c4450622545a1826"} Jul 09 04:06:54.276451 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:54.276409 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bv4mn" event={"ID":"889db0de-781d-43d6-9df1-7a9eb15a1794","Type":"ContainerStarted","Data":"0cea159a8261a7cb57a82178727c37939f9c4f7fd5b7a74c9240fe9d72bf9525"} Jul 09 04:06:54.290839 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:54.290809 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" event={"ID":"9d4cab2f-0478-4c53-9966-d96d5e18f2dd","Type":"ContainerStarted","Data":"1ce95a56b149dd15f614640bf0f5607f033ef5ae34bcaa55b3925f210d0278e4"} Jul 09 04:06:54.300794 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:54.300761 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-153.ec2.internal" event={"ID":"67b85ef8254593a9d9a03d6a7f9822a9","Type":"ContainerStarted","Data":"2de79f847bed4ef908658dc9053d939d06ae63949d42000bd11168e26048b885"} Jul 09 04:06:54.317708 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:54.317653 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-153.ec2.internal" podStartSLOduration=2.317633949 podStartE2EDuration="2.317633949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:06:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:06:54.317536271 +0000 UTC m=+4.541154757" watchObservedRunningTime="2026-07-09 04:06:54.317633949 +0000 UTC m=+4.541252445" Jul 09 04:06:54.321436 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:54.320770 2534 generic.go:356] "Generic (PLEG): container finished" podID="e28bfc992df73a71242390a6bf756269" containerID="50f02128d1a6664dda41db05db932707519fa4f66453c07a00cb666e81ad5f8f" exitCode=0 Jul 09 04:06:54.321436 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:54.320806 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-153.ec2.internal" event={"ID":"e28bfc992df73a71242390a6bf756269","Type":"ContainerDied","Data":"50f02128d1a6664dda41db05db932707519fa4f66453c07a00cb666e81ad5f8f"} Jul 09 04:06:55.245196 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:55.244801 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:06:55.245196 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:55.244950 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4gcd" podUID="9fefcefc-4128-4422-98de-b17e5949cfc7" Jul 09 04:06:55.245762 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:55.245360 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:06:55.245762 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:55.245455 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xpfm" podUID="8e433de1-ad41-4547-b130-e06490dba1bb" Jul 09 04:06:55.345878 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:55.345464 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-153.ec2.internal" event={"ID":"e28bfc992df73a71242390a6bf756269","Type":"ContainerStarted","Data":"87d23be2edecc946fadabe491c67d95063862d0105d7755ae18b90c981585b26"} Jul 09 04:06:55.783180 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:55.783138 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-original-pull-secret\") pod \"global-pull-secret-syncer-krxxf\" (UID: \"2c0aaa25-8a07-4da9-8d7f-b8abd6026f36\") " pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:06:55.783410 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:55.783305 2534 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 04:06:55.783410 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:55.783390 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-original-pull-secret podName:2c0aaa25-8a07-4da9-8d7f-b8abd6026f36 nodeName:}" failed. No retries permitted until 2026-07-09 04:06:59.783369601 +0000 UTC m=+10.006988074 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-original-pull-secret") pod "global-pull-secret-syncer-krxxf" (UID: "2c0aaa25-8a07-4da9-8d7f-b8abd6026f36") : object "kube-system"/"original-pull-secret" not registered Jul 09 04:06:55.884793 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:55.884138 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs\") pod \"network-metrics-daemon-b4gcd\" (UID: \"9fefcefc-4128-4422-98de-b17e5949cfc7\") " pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:06:55.884793 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:55.884335 2534 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:06:55.884793 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:55.884408 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs podName:9fefcefc-4128-4422-98de-b17e5949cfc7 nodeName:}" failed. No retries permitted until 2026-07-09 04:06:59.884386943 +0000 UTC m=+10.108005422 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs") pod "network-metrics-daemon-b4gcd" (UID: "9fefcefc-4128-4422-98de-b17e5949cfc7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:06:55.985207 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:55.985164 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxdh5\" (UniqueName: \"kubernetes.io/projected/8e433de1-ad41-4547-b130-e06490dba1bb-kube-api-access-dxdh5\") pod \"network-check-target-8xpfm\" (UID: \"8e433de1-ad41-4547-b130-e06490dba1bb\") " pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:06:55.985438 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:55.985417 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 04:06:55.985507 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:55.985449 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 04:06:55.985507 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:55.985463 2534 projected.go:196] Error preparing data for projected volume kube-api-access-dxdh5 for pod openshift-network-diagnostics/network-check-target-8xpfm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:06:55.985636 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:55.985527 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e433de1-ad41-4547-b130-e06490dba1bb-kube-api-access-dxdh5 podName:8e433de1-ad41-4547-b130-e06490dba1bb nodeName:}" failed. No retries permitted until 2026-07-09 04:06:59.985505967 +0000 UTC m=+10.209124438 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dxdh5" (UniqueName: "kubernetes.io/projected/8e433de1-ad41-4547-b130-e06490dba1bb-kube-api-access-dxdh5") pod "network-check-target-8xpfm" (UID: "8e433de1-ad41-4547-b130-e06490dba1bb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:06:56.245701 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:56.245591 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:06:56.246252 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:56.245729 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krxxf" podUID="2c0aaa25-8a07-4da9-8d7f-b8abd6026f36" Jul 09 04:06:57.244689 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:57.244656 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:06:57.244887 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:57.244657 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:06:57.244887 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:57.244822 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xpfm" podUID="8e433de1-ad41-4547-b130-e06490dba1bb" Jul 09 04:06:57.245023 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:57.244890 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4gcd" podUID="9fefcefc-4128-4422-98de-b17e5949cfc7" Jul 09 04:06:58.246356 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:58.245486 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:06:58.246356 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:58.246105 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krxxf" podUID="2c0aaa25-8a07-4da9-8d7f-b8abd6026f36" Jul 09 04:06:59.245142 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:59.245059 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:06:59.245338 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:59.245219 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4gcd" podUID="9fefcefc-4128-4422-98de-b17e5949cfc7" Jul 09 04:06:59.245338 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:59.245297 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:06:59.245470 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:59.245422 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xpfm" podUID="8e433de1-ad41-4547-b130-e06490dba1bb" Jul 09 04:06:59.828001 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:59.827956 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-original-pull-secret\") pod \"global-pull-secret-syncer-krxxf\" (UID: \"2c0aaa25-8a07-4da9-8d7f-b8abd6026f36\") " pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:06:59.828485 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:59.828199 2534 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 04:06:59.828485 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:59.828284 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-original-pull-secret podName:2c0aaa25-8a07-4da9-8d7f-b8abd6026f36 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:07.828268559 +0000 UTC m=+18.051887041 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-original-pull-secret") pod "global-pull-secret-syncer-krxxf" (UID: "2c0aaa25-8a07-4da9-8d7f-b8abd6026f36") : object "kube-system"/"original-pull-secret" not registered Jul 09 04:06:59.928636 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:06:59.928600 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs\") pod \"network-metrics-daemon-b4gcd\" (UID: \"9fefcefc-4128-4422-98de-b17e5949cfc7\") " pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:06:59.928831 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:59.928801 2534 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:06:59.928887 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:06:59.928876 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs podName:9fefcefc-4128-4422-98de-b17e5949cfc7 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:07.928853148 +0000 UTC m=+18.152471625 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs") pod "network-metrics-daemon-b4gcd" (UID: "9fefcefc-4128-4422-98de-b17e5949cfc7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:07:00.029439 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:00.029406 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxdh5\" (UniqueName: \"kubernetes.io/projected/8e433de1-ad41-4547-b130-e06490dba1bb-kube-api-access-dxdh5\") pod \"network-check-target-8xpfm\" (UID: \"8e433de1-ad41-4547-b130-e06490dba1bb\") " pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:07:00.029597 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:00.029585 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 04:07:00.029653 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:00.029606 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 04:07:00.029653 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:00.029615 2534 projected.go:196] Error preparing data for projected volume kube-api-access-dxdh5 for pod openshift-network-diagnostics/network-check-target-8xpfm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:07:00.029724 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:00.029667 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e433de1-ad41-4547-b130-e06490dba1bb-kube-api-access-dxdh5 podName:8e433de1-ad41-4547-b130-e06490dba1bb nodeName:}" failed. No retries permitted until 2026-07-09 04:07:08.029652063 +0000 UTC m=+18.253270530 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dxdh5" (UniqueName: "kubernetes.io/projected/8e433de1-ad41-4547-b130-e06490dba1bb-kube-api-access-dxdh5") pod "network-check-target-8xpfm" (UID: "8e433de1-ad41-4547-b130-e06490dba1bb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:07:00.248086 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:00.247999 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:07:00.248275 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:00.248137 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krxxf" podUID="2c0aaa25-8a07-4da9-8d7f-b8abd6026f36" Jul 09 04:07:01.243990 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:01.243955 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:07:01.244529 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:01.244082 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4gcd" podUID="9fefcefc-4128-4422-98de-b17e5949cfc7" Jul 09 04:07:01.244529 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:01.244447 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:07:01.244529 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:01.244523 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xpfm" podUID="8e433de1-ad41-4547-b130-e06490dba1bb" Jul 09 04:07:02.246436 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:02.246394 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:07:02.246927 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:02.246538 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krxxf" podUID="2c0aaa25-8a07-4da9-8d7f-b8abd6026f36" Jul 09 04:07:03.245107 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:03.244589 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:07:03.245107 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:03.244618 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:07:03.245107 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:03.244737 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4gcd" podUID="9fefcefc-4128-4422-98de-b17e5949cfc7" Jul 09 04:07:03.245107 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:03.244885 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xpfm" podUID="8e433de1-ad41-4547-b130-e06490dba1bb" Jul 09 04:07:04.244883 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:04.244851 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:07:04.245403 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:04.244994 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krxxf" podUID="2c0aaa25-8a07-4da9-8d7f-b8abd6026f36" Jul 09 04:07:05.244174 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:05.244095 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:07:05.244362 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:05.244222 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xpfm" podUID="8e433de1-ad41-4547-b130-e06490dba1bb" Jul 09 04:07:05.244362 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:05.244277 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:07:05.244467 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:05.244426 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4gcd" podUID="9fefcefc-4128-4422-98de-b17e5949cfc7" Jul 09 04:07:06.244274 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:06.244240 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:07:06.244681 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:06.244398 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krxxf" podUID="2c0aaa25-8a07-4da9-8d7f-b8abd6026f36" Jul 09 04:07:07.244823 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:07.244782 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:07:07.245304 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:07.244973 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4gcd" podUID="9fefcefc-4128-4422-98de-b17e5949cfc7" Jul 09 04:07:07.245304 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:07.244993 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:07:07.245304 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:07.245086 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xpfm" podUID="8e433de1-ad41-4547-b130-e06490dba1bb" Jul 09 04:07:07.897047 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:07.897015 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-original-pull-secret\") pod \"global-pull-secret-syncer-krxxf\" (UID: \"2c0aaa25-8a07-4da9-8d7f-b8abd6026f36\") " pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:07:07.897242 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:07.897220 2534 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 04:07:07.897331 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:07.897317 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-original-pull-secret podName:2c0aaa25-8a07-4da9-8d7f-b8abd6026f36 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:23.897295904 +0000 UTC m=+34.120914392 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-original-pull-secret") pod "global-pull-secret-syncer-krxxf" (UID: "2c0aaa25-8a07-4da9-8d7f-b8abd6026f36") : object "kube-system"/"original-pull-secret" not registered Jul 09 04:07:07.997920 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:07.997883 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs\") pod \"network-metrics-daemon-b4gcd\" (UID: \"9fefcefc-4128-4422-98de-b17e5949cfc7\") " pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:07:07.998103 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:07.998015 2534 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:07:07.998103 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:07.998083 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs podName:9fefcefc-4128-4422-98de-b17e5949cfc7 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:23.998065648 +0000 UTC m=+34.221684115 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs") pod "network-metrics-daemon-b4gcd" (UID: "9fefcefc-4128-4422-98de-b17e5949cfc7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:07:08.099158 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:08.099120 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxdh5\" (UniqueName: \"kubernetes.io/projected/8e433de1-ad41-4547-b130-e06490dba1bb-kube-api-access-dxdh5\") pod \"network-check-target-8xpfm\" (UID: \"8e433de1-ad41-4547-b130-e06490dba1bb\") " pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:07:08.099379 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:08.099315 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 04:07:08.099379 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:08.099340 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 04:07:08.099379 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:08.099353 2534 projected.go:196] Error preparing data for projected volume kube-api-access-dxdh5 for pod openshift-network-diagnostics/network-check-target-8xpfm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:07:08.099542 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:08.099415 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e433de1-ad41-4547-b130-e06490dba1bb-kube-api-access-dxdh5 podName:8e433de1-ad41-4547-b130-e06490dba1bb nodeName:}" failed. No retries permitted until 2026-07-09 04:07:24.099395727 +0000 UTC m=+34.323014211 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dxdh5" (UniqueName: "kubernetes.io/projected/8e433de1-ad41-4547-b130-e06490dba1bb-kube-api-access-dxdh5") pod "network-check-target-8xpfm" (UID: "8e433de1-ad41-4547-b130-e06490dba1bb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:07:08.245045 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:08.244964 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:07:08.245467 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:08.245089 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krxxf" podUID="2c0aaa25-8a07-4da9-8d7f-b8abd6026f36" Jul 09 04:07:09.244943 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:09.244906 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:07:09.245115 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:09.244906 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:07:09.245115 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:09.245065 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4gcd" podUID="9fefcefc-4128-4422-98de-b17e5949cfc7" Jul 09 04:07:09.245523 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:09.245159 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xpfm" podUID="8e433de1-ad41-4547-b130-e06490dba1bb" Jul 09 04:07:10.246231 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:10.245760 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:07:10.246231 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:10.245872 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krxxf" podUID="2c0aaa25-8a07-4da9-8d7f-b8abd6026f36" Jul 09 04:07:11.244010 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:11.243968 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:07:11.244168 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:11.244096 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4gcd" podUID="9fefcefc-4128-4422-98de-b17e5949cfc7" Jul 09 04:07:11.244259 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:11.243968 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:07:11.244371 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:11.244353 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xpfm" podUID="8e433de1-ad41-4547-b130-e06490dba1bb" Jul 09 04:07:12.244210 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:12.244174 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:07:12.244706 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:12.244322 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krxxf" podUID="2c0aaa25-8a07-4da9-8d7f-b8abd6026f36" Jul 09 04:07:13.244945 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:13.244686 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:07:13.245635 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:13.245029 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xpfm" podUID="8e433de1-ad41-4547-b130-e06490dba1bb" Jul 09 04:07:13.245635 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:13.244728 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:07:13.245635 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:13.245455 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4gcd" podUID="9fefcefc-4128-4422-98de-b17e5949cfc7" Jul 09 04:07:13.387078 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:13.387049 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xksp2" event={"ID":"e6cff921-2b0d-4748-ab0f-642fc9ff3d7e","Type":"ContainerStarted","Data":"c5e4835e8a91f9579d82210c5f3b85c9c3726172082cf1b6e31f813b84d3184e"} Jul 09 04:07:13.388894 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:13.388872 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" event={"ID":"4b157285-2103-4dc1-b461-ac80ec29eb97","Type":"ContainerStarted","Data":"bc714506cfd9f717e9144b6d134ff1767a24a48a69db4fcaaa04bf1cb196efce"} Jul 09 04:07:13.389004 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:13.388940 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" event={"ID":"4b157285-2103-4dc1-b461-ac80ec29eb97","Type":"ContainerStarted","Data":"05eaaaa2a92951457bde4e28f4683e9f449a37b8495364a375ecae1c12cad38b"} Jul 09 04:07:13.389004 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:13.388982 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" event={"ID":"4b157285-2103-4dc1-b461-ac80ec29eb97","Type":"ContainerStarted","Data":"9603f38ca05fc8af5ebe4ab0ba4866ec20156e6b911769ae2b3dc1c325a2fd73"} Jul 09 04:07:13.390244 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:13.390226 2534 generic.go:356] "Generic (PLEG): container finished" podID="ee4f80f2-7676-489e-9fe0-99c52e384790" containerID="b6e9e4df4f66aceff1cd179e4947299933601e130fec4053f839730f4f906bba" exitCode=0 Jul 09 04:07:13.390327 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:13.390300 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k9th4" event={"ID":"ee4f80f2-7676-489e-9fe0-99c52e384790","Type":"ContainerDied","Data":"b6e9e4df4f66aceff1cd179e4947299933601e130fec4053f839730f4f906bba"} Jul 09 04:07:13.391742 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:13.391548 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dgz8s" event={"ID":"0a7d54c6-9262-4462-8b4b-f35a678e2ddf","Type":"ContainerStarted","Data":"1e33e4392f82471c2e843c333009f0748f4d70ab3884995837218d98ad0f4685"} Jul 09 04:07:13.392944 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:13.392865 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ks45b" event={"ID":"758a58c5-cb89-447a-b25a-6c3d87b1ba11","Type":"ContainerStarted","Data":"00e2ad82c67927848200601e3b9a3735d8afb5ffdd6119fe273c80ac9839e79e"} Jul 09 04:07:13.394292 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:13.394255 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zpr9q" event={"ID":"8104c5b9-011c-4e10-9a15-53bdd52870f0","Type":"ContainerStarted","Data":"faee49956cdf288725de91ccbab40d97a32dd659b32d5ffeae02107457bd302b"} Jul 09 04:07:13.395695 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:13.395662 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bv4mn" event={"ID":"889db0de-781d-43d6-9df1-7a9eb15a1794","Type":"ContainerStarted","Data":"dfe99910d5c3d57e2d7c023e317bd0c00f96dcbdf315a3c97e48765634cfdb5d"} Jul 09 04:07:13.396947 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:13.396927 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" event={"ID":"9d4cab2f-0478-4c53-9966-d96d5e18f2dd","Type":"ContainerStarted","Data":"50850cf7bb0c405d14b1f958a5835b3d31f64a64cf63d61d6f0cc8b69d0c827e"} Jul 09 04:07:13.411213 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:13.411179 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-153.ec2.internal" podStartSLOduration=21.411167168 podStartE2EDuration="21.411167168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:06:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:06:55.362770752 +0000 UTC m=+5.586389245" watchObservedRunningTime="2026-07-09 04:07:13.411167168 +0000 UTC m=+23.634785657" Jul 09 04:07:13.427498 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:13.427463 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bv4mn" podStartSLOduration=3.030429341 podStartE2EDuration="22.427452593s" totalImagesPullingTime="19.397023252s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:06:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:13.426959188 +0000 UTC m=+23.650577676" watchObservedRunningTime="2026-07-09 04:07:13.427452593 +0000 UTC m=+23.651071080" Jul 09 04:07:13.444983 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:13.444950 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zpr9q" podStartSLOduration=1.937022362 podStartE2EDuration="22.444937241s" totalImagesPullingTime="20.507914879s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:06:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:13.444390514 +0000 UTC m=+23.668009003" watchObservedRunningTime="2026-07-09 04:07:13.444937241 +0000 UTC m=+23.668555729" Jul 09 04:07:13.456802 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:13.456768 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dgz8s" podStartSLOduration=1.968003574 podStartE2EDuration="22.456756042s" totalImagesPullingTime="20.488752468s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:06:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:13.456691118 +0000 UTC m=+23.680309608" watchObservedRunningTime="2026-07-09 04:07:13.456756042 +0000 UTC m=+23.680374530" Jul 09 04:07:13.472841 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:13.472762 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ks45b" podStartSLOduration=1.9745400210000001 podStartE2EDuration="22.472754448s" totalImagesPullingTime="20.498214427s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:06:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:13.47242446 +0000 UTC m=+23.696042948" watchObservedRunningTime="2026-07-09 04:07:13.472754448 +0000 UTC m=+23.696372935" Jul 09 04:07:13.490483 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:13.490453 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xksp2" podStartSLOduration=2.024782013 podStartE2EDuration="22.490441879s" totalImagesPullingTime="20.465659866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:06:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:13.490399305 +0000 UTC m=+23.714017790" watchObservedRunningTime="2026-07-09 04:07:13.490441879 +0000 UTC m=+23.714060366" Jul 09 04:07:14.244317 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:14.244268 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:07:14.244444 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:14.244396 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krxxf" podUID="2c0aaa25-8a07-4da9-8d7f-b8abd6026f36" Jul 09 04:07:14.401483 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:14.401451 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rh79z" event={"ID":"7867a42f-0f05-4c3c-8e48-fa68962734c1","Type":"ContainerStarted","Data":"19c1ecea686eaaf12ccc0095257ff8d5f259264e17b062e6dd5f06f16bc8eef3"} Jul 09 04:07:14.405184 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:14.405162 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" event={"ID":"4b157285-2103-4dc1-b461-ac80ec29eb97","Type":"ContainerStarted","Data":"46e32ca8f2a6bd422a7193f1d39d562a1d8c74b1f0eb76c8039e15e4c5c7dc89"} Jul 09 04:07:14.405298 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:14.405220 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" event={"ID":"4b157285-2103-4dc1-b461-ac80ec29eb97","Type":"ContainerStarted","Data":"fc358206e9073cbfd9eb6757db2b6f64afc8191d1e5cdb356958f60a0c39c9cc"} Jul 09 04:07:14.405298 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:14.405251 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" event={"ID":"4b157285-2103-4dc1-b461-ac80ec29eb97","Type":"ContainerStarted","Data":"326227d8aaf93c592dbed4e85ba1f874c96369aba1ff4214eff5d06d8861336f"} Jul 09 04:07:14.481804 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:14.481780 2534 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 04:07:15.202725 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:15.202627 2534 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T04:07:14.4817983Z","UUID":"276450df-e3f7-4dc5-9021-369e294dfebe","Handler":null,"Name":"","Endpoint":""} Jul 09 04:07:15.205402 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:15.205030 2534 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 04:07:15.205402 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:15.205086 2534 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 04:07:15.244714 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:15.244690 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:07:15.244968 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:15.244802 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:07:15.244968 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:15.244803 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xpfm" podUID="8e433de1-ad41-4547-b130-e06490dba1bb" Jul 09 04:07:15.244968 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:15.244887 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4gcd" podUID="9fefcefc-4128-4422-98de-b17e5949cfc7" Jul 09 04:07:15.409882 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:15.409437 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" event={"ID":"9d4cab2f-0478-4c53-9966-d96d5e18f2dd","Type":"ContainerStarted","Data":"8b2b2c3fec73afc85889858b966741a9c8e59f86df9a10bc1a3f50865f33cd6b"} Jul 09 04:07:16.244196 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:16.244128 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:07:16.244379 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:16.244267 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krxxf" podUID="2c0aaa25-8a07-4da9-8d7f-b8abd6026f36" Jul 09 04:07:16.413701 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:16.413661 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" event={"ID":"9d4cab2f-0478-4c53-9966-d96d5e18f2dd","Type":"ContainerStarted","Data":"f0524a45db5373d4867f3431affd7afec614e4b466a1170038122acb9bfc0e9d"} Jul 09 04:07:16.417046 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:16.417022 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" event={"ID":"4b157285-2103-4dc1-b461-ac80ec29eb97","Type":"ContainerStarted","Data":"6b8b5d33fc54cc4b1426c8d9af1b7c61523d228eb5d1cc6cbfc5062214c5081a"} Jul 09 04:07:16.430470 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:16.430296 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rh79z" podStartSLOduration=6.07940624 podStartE2EDuration="25.430265684s" totalImagesPullingTime="19.350859444s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:06:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:14.444502205 +0000 UTC m=+24.668120694" watchObservedRunningTime="2026-07-09 04:07:16.430265684 +0000 UTC m=+26.653884172" Jul 09 04:07:16.430594 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:16.430576 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-58mgf" podStartSLOduration=3.599533456 podStartE2EDuration="25.430571087s" totalImagesPullingTime="21.831037631s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:06:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:16.430108651 +0000 UTC m=+26.653727139" watchObservedRunningTime="2026-07-09 04:07:16.430571087 +0000 UTC m=+26.654189575" Jul 09 04:07:17.244236 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:17.244193 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:07:17.244236 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:17.244228 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:07:17.244519 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:17.244355 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4gcd" podUID="9fefcefc-4128-4422-98de-b17e5949cfc7" Jul 09 04:07:17.244519 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:17.244479 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xpfm" podUID="8e433de1-ad41-4547-b130-e06490dba1bb" Jul 09 04:07:17.257559 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:17.257531 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xksp2" Jul 09 04:07:17.258230 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:17.258207 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xksp2" Jul 09 04:07:17.420174 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:17.420139 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xksp2" Jul 09 04:07:17.420789 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:17.420769 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xksp2" Jul 09 04:07:18.244224 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:18.244159 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:07:18.244365 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:18.244318 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krxxf" podUID="2c0aaa25-8a07-4da9-8d7f-b8abd6026f36" Jul 09 04:07:18.427025 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:18.426782 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" event={"ID":"4b157285-2103-4dc1-b461-ac80ec29eb97","Type":"ContainerStarted","Data":"0ddf333cddcbf9d7d962efeb86e303579c84e34f981eb0285e6435c971839102"} Jul 09 04:07:18.453521 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:18.453474 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" podStartSLOduration=6.91086524 podStartE2EDuration="27.453459334s" totalImagesPullingTime="20.542594094s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:06:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:18.452962425 +0000 UTC m=+28.676580923" watchObservedRunningTime="2026-07-09 04:07:18.453459334 +0000 UTC m=+28.677077821" Jul 09 04:07:19.244137 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:19.244023 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:07:19.244137 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:19.244030 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:07:19.244356 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:19.244151 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xpfm" podUID="8e433de1-ad41-4547-b130-e06490dba1bb" Jul 09 04:07:19.244356 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:19.244315 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4gcd" podUID="9fefcefc-4128-4422-98de-b17e5949cfc7" Jul 09 04:07:19.431420 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:19.431384 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:07:19.431876 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:19.431475 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:07:19.431876 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:19.431528 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:07:19.450080 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:19.449812 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:07:19.466573 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:19.466542 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:07:20.124221 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:20.124191 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8xpfm"] Jul 09 04:07:20.124413 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:20.124322 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:07:20.124477 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:20.124431 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xpfm" podUID="8e433de1-ad41-4547-b130-e06490dba1bb" Jul 09 04:07:20.126945 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:20.126912 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-krxxf"] Jul 09 04:07:20.127076 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:20.127019 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:07:20.127135 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:20.127119 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krxxf" podUID="2c0aaa25-8a07-4da9-8d7f-b8abd6026f36" Jul 09 04:07:20.127578 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:20.127550 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b4gcd"] Jul 09 04:07:20.127692 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:20.127646 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:07:20.127744 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:20.127724 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4gcd" podUID="9fefcefc-4128-4422-98de-b17e5949cfc7" Jul 09 04:07:20.435219 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:20.435123 2534 generic.go:356] "Generic (PLEG): container finished" podID="ee4f80f2-7676-489e-9fe0-99c52e384790" containerID="c991c558d8511890526abbcb8a2eac95ab0c913fdd9eff0c6cb1aa1f4ed214e3" exitCode=0 Jul 09 04:07:20.435219 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:20.435186 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k9th4" event={"ID":"ee4f80f2-7676-489e-9fe0-99c52e384790","Type":"ContainerDied","Data":"c991c558d8511890526abbcb8a2eac95ab0c913fdd9eff0c6cb1aa1f4ed214e3"} Jul 09 04:07:22.244191 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:22.244161 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:07:22.244600 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:22.244161 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:07:22.244600 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:22.244317 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4gcd" podUID="9fefcefc-4128-4422-98de-b17e5949cfc7" Jul 09 04:07:22.244600 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:22.244161 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:07:22.244600 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:22.244356 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krxxf" podUID="2c0aaa25-8a07-4da9-8d7f-b8abd6026f36" Jul 09 04:07:22.244600 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:22.244411 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xpfm" podUID="8e433de1-ad41-4547-b130-e06490dba1bb" Jul 09 04:07:22.442530 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:22.442446 2534 generic.go:356] "Generic (PLEG): container finished" podID="ee4f80f2-7676-489e-9fe0-99c52e384790" containerID="32b154e75277233d15837bffaa4c843319d93d542dc5eba924d1d16e4b7cc630" exitCode=0 Jul 09 04:07:22.442530 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:22.442517 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k9th4" event={"ID":"ee4f80f2-7676-489e-9fe0-99c52e384790","Type":"ContainerDied","Data":"32b154e75277233d15837bffaa4c843319d93d542dc5eba924d1d16e4b7cc630"} Jul 09 04:07:23.916193 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:23.916022 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-original-pull-secret\") pod \"global-pull-secret-syncer-krxxf\" (UID: \"2c0aaa25-8a07-4da9-8d7f-b8abd6026f36\") " pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:07:23.916512 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:23.916198 2534 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 04:07:23.916512 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:23.916277 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-original-pull-secret podName:2c0aaa25-8a07-4da9-8d7f-b8abd6026f36 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:55.916256826 +0000 UTC m=+66.139875296 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-original-pull-secret") pod "global-pull-secret-syncer-krxxf" (UID: "2c0aaa25-8a07-4da9-8d7f-b8abd6026f36") : object "kube-system"/"original-pull-secret" not registered Jul 09 04:07:24.017296 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:24.017244 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs\") pod \"network-metrics-daemon-b4gcd\" (UID: \"9fefcefc-4128-4422-98de-b17e5949cfc7\") " pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:07:24.017434 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:24.017386 2534 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:07:24.017470 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:24.017440 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs podName:9fefcefc-4128-4422-98de-b17e5949cfc7 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:56.017425839 +0000 UTC m=+66.241044304 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs") pod "network-metrics-daemon-b4gcd" (UID: "9fefcefc-4128-4422-98de-b17e5949cfc7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:07:24.118490 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:24.118457 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxdh5\" (UniqueName: \"kubernetes.io/projected/8e433de1-ad41-4547-b130-e06490dba1bb-kube-api-access-dxdh5\") pod \"network-check-target-8xpfm\" (UID: \"8e433de1-ad41-4547-b130-e06490dba1bb\") " pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:07:24.118643 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:24.118626 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 04:07:24.118680 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:24.118648 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 04:07:24.118680 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:24.118658 2534 projected.go:196] Error preparing data for projected volume kube-api-access-dxdh5 for pod openshift-network-diagnostics/network-check-target-8xpfm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:07:24.118757 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:24.118711 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e433de1-ad41-4547-b130-e06490dba1bb-kube-api-access-dxdh5 podName:8e433de1-ad41-4547-b130-e06490dba1bb nodeName:}" failed. No retries permitted until 2026-07-09 04:07:56.11869637 +0000 UTC m=+66.342314843 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-dxdh5" (UniqueName: "kubernetes.io/projected/8e433de1-ad41-4547-b130-e06490dba1bb-kube-api-access-dxdh5") pod "network-check-target-8xpfm" (UID: "8e433de1-ad41-4547-b130-e06490dba1bb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:07:24.244510 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:24.244438 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:07:24.244510 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:24.244452 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:07:24.244510 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:24.244438 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:07:24.244708 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:24.244539 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4gcd" podUID="9fefcefc-4128-4422-98de-b17e5949cfc7" Jul 09 04:07:24.244708 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:24.244595 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xpfm" podUID="8e433de1-ad41-4547-b130-e06490dba1bb" Jul 09 04:07:24.244708 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:24.244665 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krxxf" podUID="2c0aaa25-8a07-4da9-8d7f-b8abd6026f36" Jul 09 04:07:24.451468 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:24.451430 2534 generic.go:356] "Generic (PLEG): container finished" podID="ee4f80f2-7676-489e-9fe0-99c52e384790" containerID="39b019b8c34cc123fff373a8752b99c3407ba2d47b342fbf12d98a21f75b927c" exitCode=0 Jul 09 04:07:24.451615 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:24.451483 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k9th4" event={"ID":"ee4f80f2-7676-489e-9fe0-99c52e384790","Type":"ContainerDied","Data":"39b019b8c34cc123fff373a8752b99c3407ba2d47b342fbf12d98a21f75b927c"} Jul 09 04:07:25.644881 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.644852 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-153.ec2.internal" event="NodeReady" Jul 09 04:07:25.645383 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.644999 2534 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 04:07:25.678779 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.678755 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6cd88777bb-gm7kj"] Jul 09 04:07:25.683172 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.682632 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:25.683360 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.683333 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w"] Jul 09 04:07:25.685149 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.685127 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 09 04:07:25.685149 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.685148 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 09 04:07:25.686843 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.686059 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 09 04:07:25.686843 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.686314 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fd6z5\"" Jul 09 04:07:25.689272 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.687624 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-5lk5q"] Jul 09 04:07:25.689272 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.687921 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" Jul 09 04:07:25.691896 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.691746 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-59nnd\"" Jul 09 04:07:25.691896 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.691753 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 09 04:07:25.691896 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.691845 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 09 04:07:25.692858 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.692839 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-5lk5q" Jul 09 04:07:25.693262 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.693246 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 09 04:07:25.697582 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.697562 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6prtw"] Jul 09 04:07:25.698871 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.698850 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 04:07:25.699016 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.698886 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-kx2lv\"" Jul 09 04:07:25.699099 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.699074 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 04:07:25.700917 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.700897 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-5lk5q"] Jul 09 04:07:25.701003 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.700932 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w"] Jul 09 04:07:25.701003 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.700950 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cd88777bb-gm7kj"] Jul 09 04:07:25.701126 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.701016 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6prtw" Jul 09 04:07:25.703337 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.703318 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 04:07:25.703600 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.703585 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 04:07:25.703677 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.703599 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mbnmf\"" Jul 09 04:07:25.707516 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.707496 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ktvhk"] Jul 09 04:07:25.712428 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.711916 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6prtw"] Jul 09 04:07:25.712428 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.712015 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ktvhk" Jul 09 04:07:25.714421 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.714402 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 04:07:25.714540 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.714448 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 04:07:25.714675 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.714658 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 04:07:25.714886 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.714870 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kr28h\"" Jul 09 04:07:25.722073 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.722054 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ktvhk"] Jul 09 04:07:25.731587 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.731539 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8b49d1ca-6e2d-4701-b291-a1a9c9810431-ca-trust-extracted\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:25.731587 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.731563 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8b49d1ca-6e2d-4701-b291-a1a9c9810431-installation-pull-secrets\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:25.731723 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.731590 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-z5c2w\" (UID: \"9d2284ee-ac3c-4e3c-9b08-67abc9f14e75\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" Jul 09 04:07:25.731723 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.731628 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/675fded2-27b6-4bda-83ad-84f46e175548-config-volume\") pod \"dns-default-6prtw\" (UID: \"675fded2-27b6-4bda-83ad-84f46e175548\") " pod="openshift-dns/dns-default-6prtw" Jul 09 04:07:25.731723 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.731650 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/675fded2-27b6-4bda-83ad-84f46e175548-tmp-dir\") pod \"dns-default-6prtw\" (UID: \"675fded2-27b6-4bda-83ad-84f46e175548\") " pod="openshift-dns/dns-default-6prtw" Jul 09 04:07:25.731723 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.731706 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b49d1ca-6e2d-4701-b291-a1a9c9810431-trusted-ca\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:25.731905 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.731740 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3eb622e3-4ce7-4fe0-a2ae-b5c83a81fde6-cert\") pod \"ingress-canary-ktvhk\" (UID: \"3eb622e3-4ce7-4fe0-a2ae-b5c83a81fde6\") " pod="openshift-ingress-canary/ingress-canary-ktvhk" Jul 09 04:07:25.731905 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.731781 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8b49d1ca-6e2d-4701-b291-a1a9c9810431-image-registry-private-configuration\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:25.731905 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.731815 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-bound-sa-token\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:25.731905 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.731845 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls\") pod \"dns-default-6prtw\" (UID: \"675fded2-27b6-4bda-83ad-84f46e175548\") " pod="openshift-dns/dns-default-6prtw" Jul 09 04:07:25.731905 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.731869 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wppkd\" (UniqueName: \"kubernetes.io/projected/361b013d-9119-4725-8154-5c1bafcfe9b9-kube-api-access-wppkd\") pod \"network-check-source-989f8b8dc-5lk5q\" (UID: \"361b013d-9119-4725-8154-5c1bafcfe9b9\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-5lk5q" Jul 09 04:07:25.731905 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.731901 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:25.732154 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.731923 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnrdf\" (UniqueName: \"kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-kube-api-access-mnrdf\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:25.732154 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.731960 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drpcb\" (UniqueName: \"kubernetes.io/projected/675fded2-27b6-4bda-83ad-84f46e175548-kube-api-access-drpcb\") pod \"dns-default-6prtw\" (UID: \"675fded2-27b6-4bda-83ad-84f46e175548\") " pod="openshift-dns/dns-default-6prtw" Jul 09 04:07:25.732154 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.732014 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-z5c2w\" (UID: \"9d2284ee-ac3c-4e3c-9b08-67abc9f14e75\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" Jul 09 04:07:25.732154 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.732053 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzz55\" (UniqueName: \"kubernetes.io/projected/3eb622e3-4ce7-4fe0-a2ae-b5c83a81fde6-kube-api-access-zzz55\") pod \"ingress-canary-ktvhk\" (UID: \"3eb622e3-4ce7-4fe0-a2ae-b5c83a81fde6\") " pod="openshift-ingress-canary/ingress-canary-ktvhk" Jul 09 04:07:25.732154 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.732085 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-certificates\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:25.833168 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.833136 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8b49d1ca-6e2d-4701-b291-a1a9c9810431-ca-trust-extracted\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:25.833168 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.833175 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8b49d1ca-6e2d-4701-b291-a1a9c9810431-installation-pull-secrets\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:25.833535 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.833204 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-z5c2w\" (UID: \"9d2284ee-ac3c-4e3c-9b08-67abc9f14e75\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" Jul 09 04:07:25.833535 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.833247 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/675fded2-27b6-4bda-83ad-84f46e175548-config-volume\") pod \"dns-default-6prtw\" (UID: \"675fded2-27b6-4bda-83ad-84f46e175548\") " pod="openshift-dns/dns-default-6prtw" Jul 09 04:07:25.833535 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.833269 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/675fded2-27b6-4bda-83ad-84f46e175548-tmp-dir\") pod \"dns-default-6prtw\" (UID: \"675fded2-27b6-4bda-83ad-84f46e175548\") " pod="openshift-dns/dns-default-6prtw" Jul 09 04:07:25.833535 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.833339 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b49d1ca-6e2d-4701-b291-a1a9c9810431-trusted-ca\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:25.833535 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.833378 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3eb622e3-4ce7-4fe0-a2ae-b5c83a81fde6-cert\") pod \"ingress-canary-ktvhk\" (UID: \"3eb622e3-4ce7-4fe0-a2ae-b5c83a81fde6\") " pod="openshift-ingress-canary/ingress-canary-ktvhk" Jul 09 04:07:25.833535 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.833429 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8b49d1ca-6e2d-4701-b291-a1a9c9810431-image-registry-private-configuration\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:25.833535 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.833457 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-bound-sa-token\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:25.833535 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.833480 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls\") pod \"dns-default-6prtw\" (UID: \"675fded2-27b6-4bda-83ad-84f46e175548\") " pod="openshift-dns/dns-default-6prtw" Jul 09 04:07:25.833535 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.833502 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wppkd\" (UniqueName: \"kubernetes.io/projected/361b013d-9119-4725-8154-5c1bafcfe9b9-kube-api-access-wppkd\") pod \"network-check-source-989f8b8dc-5lk5q\" (UID: \"361b013d-9119-4725-8154-5c1bafcfe9b9\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-5lk5q" Jul 09 04:07:25.833535 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.833533 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:25.833991 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.833559 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnrdf\" (UniqueName: \"kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-kube-api-access-mnrdf\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:25.833991 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.833579 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drpcb\" (UniqueName: \"kubernetes.io/projected/675fded2-27b6-4bda-83ad-84f46e175548-kube-api-access-drpcb\") pod \"dns-default-6prtw\" (UID: \"675fded2-27b6-4bda-83ad-84f46e175548\") " pod="openshift-dns/dns-default-6prtw" Jul 09 04:07:25.833991 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.833606 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-z5c2w\" (UID: \"9d2284ee-ac3c-4e3c-9b08-67abc9f14e75\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" Jul 09 04:07:25.833991 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.833649 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzz55\" (UniqueName: \"kubernetes.io/projected/3eb622e3-4ce7-4fe0-a2ae-b5c83a81fde6-kube-api-access-zzz55\") pod \"ingress-canary-ktvhk\" (UID: \"3eb622e3-4ce7-4fe0-a2ae-b5c83a81fde6\") " pod="openshift-ingress-canary/ingress-canary-ktvhk" Jul 09 04:07:25.833991 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.833651 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/675fded2-27b6-4bda-83ad-84f46e175548-tmp-dir\") pod \"dns-default-6prtw\" (UID: \"675fded2-27b6-4bda-83ad-84f46e175548\") " pod="openshift-dns/dns-default-6prtw" Jul 09 04:07:25.833991 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:25.833661 2534 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:07:25.833991 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.833672 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-certificates\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:25.833991 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.833699 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8b49d1ca-6e2d-4701-b291-a1a9c9810431-ca-trust-extracted\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:25.833991 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:25.833740 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls podName:675fded2-27b6-4bda-83ad-84f46e175548 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:26.333718403 +0000 UTC m=+36.557336876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls") pod "dns-default-6prtw" (UID: "675fded2-27b6-4bda-83ad-84f46e175548") : secret "dns-default-metrics-tls" not found Jul 09 04:07:25.834423 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.834101 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-z5c2w\" (UID: \"9d2284ee-ac3c-4e3c-9b08-67abc9f14e75\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" Jul 09 04:07:25.834423 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:25.834155 2534 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 04:07:25.834423 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:25.834171 2534 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cd88777bb-gm7kj: secret "image-registry-tls" not found Jul 09 04:07:25.834423 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:25.834218 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls podName:8b49d1ca-6e2d-4701-b291-a1a9c9810431 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:26.33420174 +0000 UTC m=+36.557820218 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls") pod "image-registry-6cd88777bb-gm7kj" (UID: "8b49d1ca-6e2d-4701-b291-a1a9c9810431") : secret "image-registry-tls" not found Jul 09 04:07:25.834423 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.834229 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b49d1ca-6e2d-4701-b291-a1a9c9810431-trusted-ca\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:25.834423 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:25.834337 2534 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 04:07:25.834423 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:25.834394 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert podName:9d2284ee-ac3c-4e3c-9b08-67abc9f14e75 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:26.33437762 +0000 UTC m=+36.557996102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-z5c2w" (UID: "9d2284ee-ac3c-4e3c-9b08-67abc9f14e75") : secret "networking-console-plugin-cert" not found Jul 09 04:07:25.834672 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.834505 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/675fded2-27b6-4bda-83ad-84f46e175548-config-volume\") pod \"dns-default-6prtw\" (UID: \"675fded2-27b6-4bda-83ad-84f46e175548\") " pod="openshift-dns/dns-default-6prtw" Jul 09 04:07:25.834713 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.834669 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-certificates\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:25.837537 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.837505 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8b49d1ca-6e2d-4701-b291-a1a9c9810431-installation-pull-secrets\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:25.837537 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.837516 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8b49d1ca-6e2d-4701-b291-a1a9c9810431-image-registry-private-configuration\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:25.837537 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.837510 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3eb622e3-4ce7-4fe0-a2ae-b5c83a81fde6-cert\") pod \"ingress-canary-ktvhk\" (UID: \"3eb622e3-4ce7-4fe0-a2ae-b5c83a81fde6\") " pod="openshift-ingress-canary/ingress-canary-ktvhk" Jul 09 04:07:25.842497 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.842473 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnrdf\" (UniqueName: \"kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-kube-api-access-mnrdf\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:25.842653 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.842627 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drpcb\" (UniqueName: \"kubernetes.io/projected/675fded2-27b6-4bda-83ad-84f46e175548-kube-api-access-drpcb\") pod \"dns-default-6prtw\" (UID: \"675fded2-27b6-4bda-83ad-84f46e175548\") " pod="openshift-dns/dns-default-6prtw" Jul 09 04:07:25.842856 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.842836 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzz55\" (UniqueName: \"kubernetes.io/projected/3eb622e3-4ce7-4fe0-a2ae-b5c83a81fde6-kube-api-access-zzz55\") pod \"ingress-canary-ktvhk\" (UID: \"3eb622e3-4ce7-4fe0-a2ae-b5c83a81fde6\") " pod="openshift-ingress-canary/ingress-canary-ktvhk" Jul 09 04:07:25.842991 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.842944 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-bound-sa-token\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:25.843058 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:25.843009 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wppkd\" (UniqueName: \"kubernetes.io/projected/361b013d-9119-4725-8154-5c1bafcfe9b9-kube-api-access-wppkd\") pod \"network-check-source-989f8b8dc-5lk5q\" (UID: \"361b013d-9119-4725-8154-5c1bafcfe9b9\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-5lk5q" Jul 09 04:07:26.013032 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:26.012954 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-5lk5q" Jul 09 04:07:26.027737 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:26.027508 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ktvhk" Jul 09 04:07:26.172833 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:26.172799 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-5lk5q"] Jul 09 04:07:26.175390 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:26.175361 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ktvhk"] Jul 09 04:07:26.178733 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:07:26.178488 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod361b013d_9119_4725_8154_5c1bafcfe9b9.slice/crio-6291befab55a301bcae5ad325766a049978cd930862502a2b4db4d2cfd3206a3 WatchSource:0}: Error finding container 6291befab55a301bcae5ad325766a049978cd930862502a2b4db4d2cfd3206a3: Status 404 returned error can't find the container with id 6291befab55a301bcae5ad325766a049978cd930862502a2b4db4d2cfd3206a3 Jul 09 04:07:26.179015 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:07:26.178975 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eb622e3_4ce7_4fe0_a2ae_b5c83a81fde6.slice/crio-2a3126a12250ab108e124df6bc49b0968f0dcd322162d43650bbaf266d024ace WatchSource:0}: Error finding container 2a3126a12250ab108e124df6bc49b0968f0dcd322162d43650bbaf266d024ace: Status 404 returned error can't find the container with id 2a3126a12250ab108e124df6bc49b0968f0dcd322162d43650bbaf266d024ace Jul 09 04:07:26.244967 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:26.244939 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:07:26.245104 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:26.244973 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:07:26.245104 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:26.244944 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:07:26.247620 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:26.247598 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 04:07:26.247757 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:26.247620 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v2z2w\"" Jul 09 04:07:26.247757 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:26.247620 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kh5dc\"" Jul 09 04:07:26.247757 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:26.247697 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 04:07:26.338596 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:26.338565 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls\") pod \"dns-default-6prtw\" (UID: \"675fded2-27b6-4bda-83ad-84f46e175548\") " pod="openshift-dns/dns-default-6prtw" Jul 09 04:07:26.338747 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:26.338603 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:26.338747 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:26.338630 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-z5c2w\" (UID: \"9d2284ee-ac3c-4e3c-9b08-67abc9f14e75\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" Jul 09 04:07:26.338747 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:26.338726 2534 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 04:07:26.338747 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:26.338733 2534 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:07:26.338747 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:26.338740 2534 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cd88777bb-gm7kj: secret "image-registry-tls" not found Jul 09 04:07:26.339007 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:26.338787 2534 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 04:07:26.339007 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:26.338799 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls podName:675fded2-27b6-4bda-83ad-84f46e175548 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:27.338777635 +0000 UTC m=+37.562396121 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls") pod "dns-default-6prtw" (UID: "675fded2-27b6-4bda-83ad-84f46e175548") : secret "dns-default-metrics-tls" not found Jul 09 04:07:26.339007 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:26.338820 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls podName:8b49d1ca-6e2d-4701-b291-a1a9c9810431 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:27.338810472 +0000 UTC m=+37.562428945 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls") pod "image-registry-6cd88777bb-gm7kj" (UID: "8b49d1ca-6e2d-4701-b291-a1a9c9810431") : secret "image-registry-tls" not found Jul 09 04:07:26.339007 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:26.338846 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert podName:9d2284ee-ac3c-4e3c-9b08-67abc9f14e75 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:27.338827362 +0000 UTC m=+37.562445848 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-z5c2w" (UID: "9d2284ee-ac3c-4e3c-9b08-67abc9f14e75") : secret "networking-console-plugin-cert" not found Jul 09 04:07:26.457674 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:26.457629 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ktvhk" event={"ID":"3eb622e3-4ce7-4fe0-a2ae-b5c83a81fde6","Type":"ContainerStarted","Data":"2a3126a12250ab108e124df6bc49b0968f0dcd322162d43650bbaf266d024ace"} Jul 09 04:07:26.458786 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:26.458758 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-5lk5q" event={"ID":"361b013d-9119-4725-8154-5c1bafcfe9b9","Type":"ContainerStarted","Data":"6291befab55a301bcae5ad325766a049978cd930862502a2b4db4d2cfd3206a3"} Jul 09 04:07:27.346020 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:27.345719 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls\") pod \"dns-default-6prtw\" (UID: \"675fded2-27b6-4bda-83ad-84f46e175548\") " pod="openshift-dns/dns-default-6prtw" Jul 09 04:07:27.346020 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:27.345779 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:27.346020 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:27.345818 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-z5c2w\" (UID: \"9d2284ee-ac3c-4e3c-9b08-67abc9f14e75\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" Jul 09 04:07:27.346020 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:27.345891 2534 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:07:27.346020 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:27.345975 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls podName:675fded2-27b6-4bda-83ad-84f46e175548 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:29.345953172 +0000 UTC m=+39.569571645 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls") pod "dns-default-6prtw" (UID: "675fded2-27b6-4bda-83ad-84f46e175548") : secret "dns-default-metrics-tls" not found Jul 09 04:07:27.346020 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:27.345995 2534 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 04:07:27.347159 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:27.347130 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert podName:9d2284ee-ac3c-4e3c-9b08-67abc9f14e75 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:29.347110168 +0000 UTC m=+39.570728647 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-z5c2w" (UID: "9d2284ee-ac3c-4e3c-9b08-67abc9f14e75") : secret "networking-console-plugin-cert" not found Jul 09 04:07:27.347159 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:27.346324 2534 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 04:07:27.347386 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:27.347171 2534 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cd88777bb-gm7kj: secret "image-registry-tls" not found Jul 09 04:07:27.347386 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:27.347222 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls podName:8b49d1ca-6e2d-4701-b291-a1a9c9810431 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:29.347206658 +0000 UTC m=+39.570825128 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls") pod "image-registry-6cd88777bb-gm7kj" (UID: "8b49d1ca-6e2d-4701-b291-a1a9c9810431") : secret "image-registry-tls" not found Jul 09 04:07:29.362786 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:29.362745 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls\") pod \"dns-default-6prtw\" (UID: \"675fded2-27b6-4bda-83ad-84f46e175548\") " pod="openshift-dns/dns-default-6prtw" Jul 09 04:07:29.363248 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:29.362802 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:29.363248 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:29.362837 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-z5c2w\" (UID: \"9d2284ee-ac3c-4e3c-9b08-67abc9f14e75\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" Jul 09 04:07:29.363248 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:29.362904 2534 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:07:29.363248 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:29.362949 2534 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 04:07:29.363248 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:29.362984 2534 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 04:07:29.363248 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:29.363002 2534 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cd88777bb-gm7kj: secret "image-registry-tls" not found Jul 09 04:07:29.363248 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:29.362986 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls podName:675fded2-27b6-4bda-83ad-84f46e175548 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:33.362970167 +0000 UTC m=+43.586588637 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls") pod "dns-default-6prtw" (UID: "675fded2-27b6-4bda-83ad-84f46e175548") : secret "dns-default-metrics-tls" not found Jul 09 04:07:29.363248 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:29.363065 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert podName:9d2284ee-ac3c-4e3c-9b08-67abc9f14e75 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:33.363045957 +0000 UTC m=+43.586664430 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-z5c2w" (UID: "9d2284ee-ac3c-4e3c-9b08-67abc9f14e75") : secret "networking-console-plugin-cert" not found Jul 09 04:07:29.363248 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:29.363089 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls podName:8b49d1ca-6e2d-4701-b291-a1a9c9810431 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:33.363078584 +0000 UTC m=+43.586697060 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls") pod "image-registry-6cd88777bb-gm7kj" (UID: "8b49d1ca-6e2d-4701-b291-a1a9c9810431") : secret "image-registry-tls" not found Jul 09 04:07:31.117027 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:31.116990 2534 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xksp2_e6cff921-2b0d-4748-ab0f-642fc9ff3d7e/konnectivity-agent/0.log" limit=1024 Jul 09 04:07:31.210565 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:31.210526 2534 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xksp2_e6cff921-2b0d-4748-ab0f-642fc9ff3d7e/konnectivity-agent/0.log" limit=1024 Jul 09 04:07:31.311413 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:31.311377 2534 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xksp2_e6cff921-2b0d-4748-ab0f-642fc9ff3d7e/konnectivity-agent/0.log" limit=1024 Jul 09 04:07:32.475276 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:32.475187 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ktvhk" event={"ID":"3eb622e3-4ce7-4fe0-a2ae-b5c83a81fde6","Type":"ContainerStarted","Data":"20d7ce27fdd70f0ef9071e8eb2e69d836cb14fd828156bf1459bc82faa68687e"} Jul 09 04:07:32.476715 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:32.476693 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-5lk5q" event={"ID":"361b013d-9119-4725-8154-5c1bafcfe9b9","Type":"ContainerStarted","Data":"e6f30c22d07b920c8b677b29cd1bc519dbdd9ed29867d88f9cd5c446724d658e"} Jul 09 04:07:32.480035 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:32.479581 2534 generic.go:356] "Generic (PLEG): container finished" podID="ee4f80f2-7676-489e-9fe0-99c52e384790" containerID="d8970809c742514bb08664bb0ccf0ff12e9d7cc5cf2556564216dca4b7d326fa" exitCode=0 Jul 09 04:07:32.480035 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:32.479621 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k9th4" event={"ID":"ee4f80f2-7676-489e-9fe0-99c52e384790","Type":"ContainerDied","Data":"d8970809c742514bb08664bb0ccf0ff12e9d7cc5cf2556564216dca4b7d326fa"} Jul 09 04:07:32.493266 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:32.493226 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ktvhk" podStartSLOduration=1.722042106 podStartE2EDuration="7.493213769s" totalImagesPullingTime="5.771171663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:07:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:32.490927379 +0000 UTC m=+42.714545867" watchObservedRunningTime="2026-07-09 04:07:32.493213769 +0000 UTC m=+42.716832257" Jul 09 04:07:32.509992 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:32.509943 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-5lk5q" podStartSLOduration=33.727527936 podStartE2EDuration="39.509925576s" totalImagesPullingTime="5.78239764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:06:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:32.508526941 +0000 UTC m=+42.732145431" watchObservedRunningTime="2026-07-09 04:07:32.509925576 +0000 UTC m=+42.733544065" Jul 09 04:07:33.395675 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:33.395643 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls\") pod \"dns-default-6prtw\" (UID: \"675fded2-27b6-4bda-83ad-84f46e175548\") " pod="openshift-dns/dns-default-6prtw" Jul 09 04:07:33.395675 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:33.395681 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:33.395912 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:33.395705 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-z5c2w\" (UID: \"9d2284ee-ac3c-4e3c-9b08-67abc9f14e75\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" Jul 09 04:07:33.395912 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:33.395789 2534 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:07:33.395912 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:33.395808 2534 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 04:07:33.395912 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:33.395816 2534 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 04:07:33.395912 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:33.395842 2534 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cd88777bb-gm7kj: secret "image-registry-tls" not found Jul 09 04:07:33.395912 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:33.395862 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert podName:9d2284ee-ac3c-4e3c-9b08-67abc9f14e75 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:41.395849081 +0000 UTC m=+51.619467550 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-z5c2w" (UID: "9d2284ee-ac3c-4e3c-9b08-67abc9f14e75") : secret "networking-console-plugin-cert" not found Jul 09 04:07:33.395912 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:33.395874 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls podName:675fded2-27b6-4bda-83ad-84f46e175548 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:41.39586883 +0000 UTC m=+51.619487297 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls") pod "dns-default-6prtw" (UID: "675fded2-27b6-4bda-83ad-84f46e175548") : secret "dns-default-metrics-tls" not found Jul 09 04:07:33.395912 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:33.395884 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls podName:8b49d1ca-6e2d-4701-b291-a1a9c9810431 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:41.395879215 +0000 UTC m=+51.619497680 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls") pod "image-registry-6cd88777bb-gm7kj" (UID: "8b49d1ca-6e2d-4701-b291-a1a9c9810431") : secret "image-registry-tls" not found Jul 09 04:07:33.485102 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:33.485066 2534 generic.go:356] "Generic (PLEG): container finished" podID="ee4f80f2-7676-489e-9fe0-99c52e384790" containerID="950b937c1cc369323bd3182b5a3691353f0e14387bc6b56ae1f0f3f21d98f9c1" exitCode=0 Jul 09 04:07:33.485556 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:33.485150 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k9th4" event={"ID":"ee4f80f2-7676-489e-9fe0-99c52e384790","Type":"ContainerDied","Data":"950b937c1cc369323bd3182b5a3691353f0e14387bc6b56ae1f0f3f21d98f9c1"} Jul 09 04:07:33.623797 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:33.623772 2534 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xksp2_e6cff921-2b0d-4748-ab0f-642fc9ff3d7e/konnectivity-agent/0.log" limit=1024 Jul 09 04:07:33.698752 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:33.698663 2534 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xksp2_e6cff921-2b0d-4748-ab0f-642fc9ff3d7e/konnectivity-agent/0.log" limit=1024 Jul 09 04:07:33.794399 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:33.794360 2534 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xksp2_e6cff921-2b0d-4748-ab0f-642fc9ff3d7e/konnectivity-agent/0.log" limit=1024 Jul 09 04:07:33.895065 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:33.895029 2534 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xksp2_e6cff921-2b0d-4748-ab0f-642fc9ff3d7e/konnectivity-agent/0.log" limit=1024 Jul 09 04:07:33.947106 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:33.947071 2534 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xksp2_e6cff921-2b0d-4748-ab0f-642fc9ff3d7e/konnectivity-agent/0.log" limit=1024 Jul 09 04:07:34.494090 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:34.494051 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k9th4" event={"ID":"ee4f80f2-7676-489e-9fe0-99c52e384790","Type":"ContainerStarted","Data":"b8a6a68a1704363aba5b6f86d2fd191c0707da74f01b5b132a03804792b7c5b4"} Jul 09 04:07:34.516420 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:34.516366 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-k9th4" podStartSLOduration=6.880637123 podStartE2EDuration="43.516350953s" totalImagesPullingTime="36.63571383s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:06:51 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:34.515960227 +0000 UTC m=+44.739578716" watchObservedRunningTime="2026-07-09 04:07:34.516350953 +0000 UTC m=+44.739969441" Jul 09 04:07:34.637304 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:34.637249 2534 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xksp2_e6cff921-2b0d-4748-ab0f-642fc9ff3d7e/konnectivity-agent/0.log" limit=1024 Jul 09 04:07:34.692538 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:34.692498 2534 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xksp2_e6cff921-2b0d-4748-ab0f-642fc9ff3d7e/konnectivity-agent/0.log" limit=1024 Jul 09 04:07:35.003609 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:35.003577 2534 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xksp2_e6cff921-2b0d-4748-ab0f-642fc9ff3d7e/konnectivity-agent/0.log" limit=1024 Jul 09 04:07:35.055656 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:35.055627 2534 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xksp2_e6cff921-2b0d-4748-ab0f-642fc9ff3d7e/konnectivity-agent/0.log" limit=1024 Jul 09 04:07:41.456239 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:41.456199 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls\") pod \"dns-default-6prtw\" (UID: \"675fded2-27b6-4bda-83ad-84f46e175548\") " pod="openshift-dns/dns-default-6prtw" Jul 09 04:07:41.456239 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:41.456242 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:41.456649 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:41.456265 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-z5c2w\" (UID: \"9d2284ee-ac3c-4e3c-9b08-67abc9f14e75\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" Jul 09 04:07:41.456649 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:41.456368 2534 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 04:07:41.456649 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:41.456375 2534 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:07:41.456649 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:41.456379 2534 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 04:07:41.456649 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:41.456418 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert podName:9d2284ee-ac3c-4e3c-9b08-67abc9f14e75 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:57.456405258 +0000 UTC m=+67.680023724 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-z5c2w" (UID: "9d2284ee-ac3c-4e3c-9b08-67abc9f14e75") : secret "networking-console-plugin-cert" not found Jul 09 04:07:41.456649 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:41.456420 2534 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cd88777bb-gm7kj: secret "image-registry-tls" not found Jul 09 04:07:41.456649 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:41.456430 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls podName:675fded2-27b6-4bda-83ad-84f46e175548 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:57.456424627 +0000 UTC m=+67.680043093 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls") pod "dns-default-6prtw" (UID: "675fded2-27b6-4bda-83ad-84f46e175548") : secret "dns-default-metrics-tls" not found Jul 09 04:07:41.456649 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:41.456450 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls podName:8b49d1ca-6e2d-4701-b291-a1a9c9810431 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:57.456435558 +0000 UTC m=+67.680054024 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls") pod "image-registry-6cd88777bb-gm7kj" (UID: "8b49d1ca-6e2d-4701-b291-a1a9c9810431") : secret "image-registry-tls" not found Jul 09 04:07:46.542199 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:46.542162 2534 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xksp2_e6cff921-2b0d-4748-ab0f-642fc9ff3d7e/konnectivity-agent/0.log" limit=1024 Jul 09 04:07:46.642250 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:46.642210 2534 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xksp2_e6cff921-2b0d-4748-ab0f-642fc9ff3d7e/konnectivity-agent/0.log" limit=1024 Jul 09 04:07:47.604163 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:47.604132 2534 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xksp2_e6cff921-2b0d-4748-ab0f-642fc9ff3d7e/konnectivity-agent/0.log" limit=1024 Jul 09 04:07:47.656649 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:47.656619 2534 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xksp2_e6cff921-2b0d-4748-ab0f-642fc9ff3d7e/konnectivity-agent/0.log" limit=1024 Jul 09 04:07:51.455313 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:51.455257 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pkb8q" Jul 09 04:07:51.604971 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:51.604933 2534 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xksp2_e6cff921-2b0d-4748-ab0f-642fc9ff3d7e/konnectivity-agent/0.log" limit=1024 Jul 09 04:07:51.655717 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:51.655684 2534 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xksp2_e6cff921-2b0d-4748-ab0f-642fc9ff3d7e/konnectivity-agent/0.log" limit=1024 Jul 09 04:07:55.902736 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:55.902696 2534 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xksp2_e6cff921-2b0d-4748-ab0f-642fc9ff3d7e/konnectivity-agent/0.log" limit=1024 Jul 09 04:07:55.970268 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:55.970224 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-original-pull-secret\") pod \"global-pull-secret-syncer-krxxf\" (UID: \"2c0aaa25-8a07-4da9-8d7f-b8abd6026f36\") " pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:07:55.972744 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:55.972722 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 04:07:55.983799 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:55.983766 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c0aaa25-8a07-4da9-8d7f-b8abd6026f36-original-pull-secret\") pod \"global-pull-secret-syncer-krxxf\" (UID: \"2c0aaa25-8a07-4da9-8d7f-b8abd6026f36\") " pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:07:56.004251 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:56.004213 2534 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xksp2_e6cff921-2b0d-4748-ab0f-642fc9ff3d7e/konnectivity-agent/0.log" limit=1024 Jul 09 04:07:56.071642 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:56.071605 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs\") pod \"network-metrics-daemon-b4gcd\" (UID: \"9fefcefc-4128-4422-98de-b17e5949cfc7\") " pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:07:56.073773 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:56.073756 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 04:07:56.082624 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:56.082601 2534 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 04:07:56.082710 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:56.082664 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs podName:9fefcefc-4128-4422-98de-b17e5949cfc7 nodeName:}" failed. No retries permitted until 2026-07-09 04:09:00.082646212 +0000 UTC m=+130.306264678 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs") pod "network-metrics-daemon-b4gcd" (UID: "9fefcefc-4128-4422-98de-b17e5949cfc7") : secret "metrics-daemon-secret" not found Jul 09 04:07:56.104994 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:56.104960 2534 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xksp2_e6cff921-2b0d-4748-ab0f-642fc9ff3d7e/konnectivity-agent/0.log" limit=1024 Jul 09 04:07:56.172523 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:56.172431 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxdh5\" (UniqueName: \"kubernetes.io/projected/8e433de1-ad41-4547-b130-e06490dba1bb-kube-api-access-dxdh5\") pod \"network-check-target-8xpfm\" (UID: \"8e433de1-ad41-4547-b130-e06490dba1bb\") " pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:07:56.175241 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:56.175218 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxdh5\" (UniqueName: \"kubernetes.io/projected/8e433de1-ad41-4547-b130-e06490dba1bb-kube-api-access-dxdh5\") pod \"network-check-target-8xpfm\" (UID: \"8e433de1-ad41-4547-b130-e06490dba1bb\") " pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:07:56.257381 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:56.257346 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krxxf" Jul 09 04:07:56.272630 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:56.272598 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kh5dc\"" Jul 09 04:07:56.279720 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:56.279692 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:07:56.386756 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:56.386727 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-krxxf"] Jul 09 04:07:56.390776 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:07:56.390746 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c0aaa25_8a07_4da9_8d7f_b8abd6026f36.slice/crio-a56a1ca0bd3c3a9950ded5960c54548b4810f98f6d1ad32e988fedbd3be824df WatchSource:0}: Error finding container a56a1ca0bd3c3a9950ded5960c54548b4810f98f6d1ad32e988fedbd3be824df: Status 404 returned error can't find the container with id a56a1ca0bd3c3a9950ded5960c54548b4810f98f6d1ad32e988fedbd3be824df Jul 09 04:07:56.415582 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:56.415554 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8xpfm"] Jul 09 04:07:56.419871 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:07:56.419834 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e433de1_ad41_4547_b130_e06490dba1bb.slice/crio-fbaac0c35079d681ea69f3aa9eab97596c02f4349efcb3266cbcd985caa93092 WatchSource:0}: Error finding container fbaac0c35079d681ea69f3aa9eab97596c02f4349efcb3266cbcd985caa93092: Status 404 returned error can't find the container with id fbaac0c35079d681ea69f3aa9eab97596c02f4349efcb3266cbcd985caa93092 Jul 09 04:07:56.546138 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:56.546104 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8xpfm" event={"ID":"8e433de1-ad41-4547-b130-e06490dba1bb","Type":"ContainerStarted","Data":"8066fdefa516149efeb26a8114b702f47bb63f18f210d61bfcdffaaabf847b2e"} Jul 09 04:07:56.546353 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:56.546150 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8xpfm" event={"ID":"8e433de1-ad41-4547-b130-e06490dba1bb","Type":"ContainerStarted","Data":"fbaac0c35079d681ea69f3aa9eab97596c02f4349efcb3266cbcd985caa93092"} Jul 09 04:07:56.546353 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:56.546221 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:07:56.547398 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:56.547378 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-krxxf" event={"ID":"2c0aaa25-8a07-4da9-8d7f-b8abd6026f36","Type":"ContainerStarted","Data":"a56a1ca0bd3c3a9950ded5960c54548b4810f98f6d1ad32e988fedbd3be824df"} Jul 09 04:07:56.560246 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:56.560201 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8xpfm" podStartSLOduration=65.560187148 podStartE2EDuration="1m5.560187148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:06:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:56.559850604 +0000 UTC m=+66.783469093" watchObservedRunningTime="2026-07-09 04:07:56.560187148 +0000 UTC m=+66.783805675" Jul 09 04:07:57.482438 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:57.482393 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls\") pod \"dns-default-6prtw\" (UID: \"675fded2-27b6-4bda-83ad-84f46e175548\") " pod="openshift-dns/dns-default-6prtw" Jul 09 04:07:57.482438 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:57.482443 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:07:57.482916 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:07:57.482466 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-z5c2w\" (UID: \"9d2284ee-ac3c-4e3c-9b08-67abc9f14e75\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" Jul 09 04:07:57.482916 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:57.482568 2534 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 04:07:57.482916 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:57.482587 2534 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 04:07:57.482916 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:57.482593 2534 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cd88777bb-gm7kj: secret "image-registry-tls" not found Jul 09 04:07:57.482916 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:57.482569 2534 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:07:57.482916 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:57.482636 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert podName:9d2284ee-ac3c-4e3c-9b08-67abc9f14e75 nodeName:}" failed. No retries permitted until 2026-07-09 04:08:29.482622721 +0000 UTC m=+99.706241188 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-z5c2w" (UID: "9d2284ee-ac3c-4e3c-9b08-67abc9f14e75") : secret "networking-console-plugin-cert" not found Jul 09 04:07:57.482916 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:57.482655 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls podName:675fded2-27b6-4bda-83ad-84f46e175548 nodeName:}" failed. No retries permitted until 2026-07-09 04:08:29.482641225 +0000 UTC m=+99.706259691 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls") pod "dns-default-6prtw" (UID: "675fded2-27b6-4bda-83ad-84f46e175548") : secret "dns-default-metrics-tls" not found Jul 09 04:07:57.482916 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:07:57.482666 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls podName:8b49d1ca-6e2d-4701-b291-a1a9c9810431 nodeName:}" failed. No retries permitted until 2026-07-09 04:08:29.482660323 +0000 UTC m=+99.706278789 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls") pod "image-registry-6cd88777bb-gm7kj" (UID: "8b49d1ca-6e2d-4701-b291-a1a9c9810431") : secret "image-registry-tls" not found Jul 09 04:08:03.567571 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:08:03.567531 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-krxxf" event={"ID":"2c0aaa25-8a07-4da9-8d7f-b8abd6026f36","Type":"ContainerStarted","Data":"057672edc05807291073cac2935635e62a6aca8e10c82ee40245bbcaef59903f"} Jul 09 04:08:03.581789 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:08:03.581736 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-krxxf" podStartSLOduration=66.005975762 podStartE2EDuration="1m12.581722384s" totalImagesPullingTime="6.575746622s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:06:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:08:03.580669855 +0000 UTC m=+73.804288343" watchObservedRunningTime="2026-07-09 04:08:03.581722384 +0000 UTC m=+73.805340873" Jul 09 04:08:27.553132 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:08:27.553101 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8xpfm" Jul 09 04:08:29.514470 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:08:29.514411 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-z5c2w\" (UID: \"9d2284ee-ac3c-4e3c-9b08-67abc9f14e75\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" Jul 09 04:08:29.514961 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:08:29.514529 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls\") pod \"dns-default-6prtw\" (UID: \"675fded2-27b6-4bda-83ad-84f46e175548\") " pod="openshift-dns/dns-default-6prtw" Jul 09 04:08:29.514961 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:08:29.514560 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:08:29.514961 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:08:29.514563 2534 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 04:08:29.514961 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:08:29.514620 2534 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:08:29.514961 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:08:29.514653 2534 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 04:08:29.514961 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:08:29.514669 2534 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cd88777bb-gm7kj: secret "image-registry-tls" not found Jul 09 04:08:29.514961 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:08:29.514678 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert podName:9d2284ee-ac3c-4e3c-9b08-67abc9f14e75 nodeName:}" failed. No retries permitted until 2026-07-09 04:09:33.514658172 +0000 UTC m=+163.738276642 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-z5c2w" (UID: "9d2284ee-ac3c-4e3c-9b08-67abc9f14e75") : secret "networking-console-plugin-cert" not found Jul 09 04:08:29.514961 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:08:29.514696 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls podName:675fded2-27b6-4bda-83ad-84f46e175548 nodeName:}" failed. No retries permitted until 2026-07-09 04:09:33.514687324 +0000 UTC m=+163.738305790 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls") pod "dns-default-6prtw" (UID: "675fded2-27b6-4bda-83ad-84f46e175548") : secret "dns-default-metrics-tls" not found Jul 09 04:08:29.514961 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:08:29.514715 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls podName:8b49d1ca-6e2d-4701-b291-a1a9c9810431 nodeName:}" failed. No retries permitted until 2026-07-09 04:09:33.514701164 +0000 UTC m=+163.738319629 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls") pod "image-registry-6cd88777bb-gm7kj" (UID: "8b49d1ca-6e2d-4701-b291-a1a9c9810431") : secret "image-registry-tls" not found Jul 09 04:09:00.128688 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:09:00.128628 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs\") pod \"network-metrics-daemon-b4gcd\" (UID: \"9fefcefc-4128-4422-98de-b17e5949cfc7\") " pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:09:00.129270 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:09:00.128826 2534 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 04:09:00.129270 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:09:00.128932 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs podName:9fefcefc-4128-4422-98de-b17e5949cfc7 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:02.128909423 +0000 UTC m=+252.352527894 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs") pod "network-metrics-daemon-b4gcd" (UID: "9fefcefc-4128-4422-98de-b17e5949cfc7") : secret "metrics-daemon-secret" not found Jul 09 04:09:28.695736 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:09:28.695678 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" podUID="8b49d1ca-6e2d-4701-b291-a1a9c9810431" Jul 09 04:09:28.705860 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:09:28.705823 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" podUID="9d2284ee-ac3c-4e3c-9b08-67abc9f14e75" Jul 09 04:09:28.721024 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:09:28.720991 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-6prtw" podUID="675fded2-27b6-4bda-83ad-84f46e175548" Jul 09 04:09:28.771593 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:09:28.771565 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:09:28.771742 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:09:28.771567 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" Jul 09 04:09:28.771742 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:09:28.771577 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6prtw" Jul 09 04:09:29.263772 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:09:29.263730 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-b4gcd" podUID="9fefcefc-4128-4422-98de-b17e5949cfc7" Jul 09 04:09:33.575530 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:09:33.575494 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls\") pod \"dns-default-6prtw\" (UID: \"675fded2-27b6-4bda-83ad-84f46e175548\") " pod="openshift-dns/dns-default-6prtw" Jul 09 04:09:33.576010 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:09:33.575540 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:09:33.576010 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:09:33.575575 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-z5c2w\" (UID: \"9d2284ee-ac3c-4e3c-9b08-67abc9f14e75\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" Jul 09 04:09:33.576010 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:09:33.575641 2534 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:09:33.576010 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:09:33.575721 2534 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 04:09:33.576010 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:09:33.575728 2534 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 04:09:33.576010 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:09:33.575738 2534 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cd88777bb-gm7kj: secret "image-registry-tls" not found Jul 09 04:09:33.576010 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:09:33.575737 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls podName:675fded2-27b6-4bda-83ad-84f46e175548 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:35.575711861 +0000 UTC m=+285.799330346 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls") pod "dns-default-6prtw" (UID: "675fded2-27b6-4bda-83ad-84f46e175548") : secret "dns-default-metrics-tls" not found Jul 09 04:09:33.576010 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:09:33.575796 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert podName:9d2284ee-ac3c-4e3c-9b08-67abc9f14e75 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:35.575777654 +0000 UTC m=+285.799396123 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-z5c2w" (UID: "9d2284ee-ac3c-4e3c-9b08-67abc9f14e75") : secret "networking-console-plugin-cert" not found Jul 09 04:09:33.576010 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:09:33.575821 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls podName:8b49d1ca-6e2d-4701-b291-a1a9c9810431 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:35.575811218 +0000 UTC m=+285.799429684 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls") pod "image-registry-6cd88777bb-gm7kj" (UID: "8b49d1ca-6e2d-4701-b291-a1a9c9810431") : secret "image-registry-tls" not found Jul 09 04:09:41.244846 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:09:41.244805 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:11:02.183191 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:02.183150 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs\") pod \"network-metrics-daemon-b4gcd\" (UID: \"9fefcefc-4128-4422-98de-b17e5949cfc7\") " pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:11:02.183713 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:02.183263 2534 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 04:11:02.183713 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:02.183342 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs podName:9fefcefc-4128-4422-98de-b17e5949cfc7 nodeName:}" failed. No retries permitted until 2026-07-09 04:13:04.183327211 +0000 UTC m=+374.406945677 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs") pod "network-metrics-daemon-b4gcd" (UID: "9fefcefc-4128-4422-98de-b17e5949cfc7") : secret "metrics-daemon-secret" not found Jul 09 04:11:18.917841 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.917807 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w48tg"] Jul 09 04:11:18.919684 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.919657 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w48tg" Jul 09 04:11:18.920106 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.920082 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-8qmfm"] Jul 09 04:11:18.921677 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.921657 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-8qmfm" Jul 09 04:11:18.922427 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.922400 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 09 04:11:18.922531 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.922508 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 09 04:11:18.922843 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.922823 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 09 04:11:18.923152 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.923130 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 09 04:11:18.923267 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.923155 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-xmktk\"" Jul 09 04:11:18.923589 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.923570 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-k8dnq"] Jul 09 04:11:18.924075 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.924058 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 09 04:11:18.924150 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.924064 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 09 04:11:18.925008 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.924991 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 04:11:18.925117 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.925089 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 04:11:18.925415 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.925388 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" Jul 09 04:11:18.926311 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.926292 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-ltgq5\"" Jul 09 04:11:18.928024 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.928007 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 09 04:11:18.928105 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.928089 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 09 04:11:18.928273 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.928256 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 09 04:11:18.928538 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.928520 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 09 04:11:18.928639 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.928588 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-7xrsx\"" Jul 09 04:11:18.930488 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.930457 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 09 04:11:18.932022 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.932000 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w48tg"] Jul 09 04:11:18.937211 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.937191 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 09 04:11:18.938339 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.938318 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-8qmfm"] Jul 09 04:11:18.941124 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.941101 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-k8dnq"] Jul 09 04:11:18.998296 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.998257 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/900fd2e6-681b-451f-8f39-92d8d13bf9e5-serving-cert\") pod \"insights-operator-5c847dcfc4-8qmfm\" (UID: \"900fd2e6-681b-451f-8f39-92d8d13bf9e5\") " pod="openshift-insights/insights-operator-5c847dcfc4-8qmfm" Jul 09 04:11:18.998474 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.998313 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8872a475-7573-4057-a22c-236c5e337a1a-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-w48tg\" (UID: \"8872a475-7573-4057-a22c-236c5e337a1a\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w48tg" Jul 09 04:11:18.998474 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.998357 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/900fd2e6-681b-451f-8f39-92d8d13bf9e5-tmp\") pod \"insights-operator-5c847dcfc4-8qmfm\" (UID: \"900fd2e6-681b-451f-8f39-92d8d13bf9e5\") " pod="openshift-insights/insights-operator-5c847dcfc4-8qmfm" Jul 09 04:11:18.998474 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.998414 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/900fd2e6-681b-451f-8f39-92d8d13bf9e5-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-8qmfm\" (UID: \"900fd2e6-681b-451f-8f39-92d8d13bf9e5\") " pod="openshift-insights/insights-operator-5c847dcfc4-8qmfm" Jul 09 04:11:18.998474 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.998435 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hchp5\" (UniqueName: \"kubernetes.io/projected/900fd2e6-681b-451f-8f39-92d8d13bf9e5-kube-api-access-hchp5\") pod \"insights-operator-5c847dcfc4-8qmfm\" (UID: \"900fd2e6-681b-451f-8f39-92d8d13bf9e5\") " pod="openshift-insights/insights-operator-5c847dcfc4-8qmfm" Jul 09 04:11:18.998474 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.998459 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/900fd2e6-681b-451f-8f39-92d8d13bf9e5-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-8qmfm\" (UID: \"900fd2e6-681b-451f-8f39-92d8d13bf9e5\") " pod="openshift-insights/insights-operator-5c847dcfc4-8qmfm" Jul 09 04:11:18.998624 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.998477 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsqn9\" (UniqueName: \"kubernetes.io/projected/8872a475-7573-4057-a22c-236c5e337a1a-kube-api-access-hsqn9\") pod \"service-ca-operator-5d46d9ffd4-w48tg\" (UID: \"8872a475-7573-4057-a22c-236c5e337a1a\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w48tg" Jul 09 04:11:18.998624 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.998503 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/633a3ae5-3d78-4217-901a-6d0d06442cf4-serving-cert\") pod \"console-operator-68b757fb67-k8dnq\" (UID: \"633a3ae5-3d78-4217-901a-6d0d06442cf4\") " pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" Jul 09 04:11:18.998624 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.998585 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/633a3ae5-3d78-4217-901a-6d0d06442cf4-config\") pod \"console-operator-68b757fb67-k8dnq\" (UID: \"633a3ae5-3d78-4217-901a-6d0d06442cf4\") " pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" Jul 09 04:11:18.998624 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.998604 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/900fd2e6-681b-451f-8f39-92d8d13bf9e5-snapshots\") pod \"insights-operator-5c847dcfc4-8qmfm\" (UID: \"900fd2e6-681b-451f-8f39-92d8d13bf9e5\") " pod="openshift-insights/insights-operator-5c847dcfc4-8qmfm" Jul 09 04:11:18.998736 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.998652 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/633a3ae5-3d78-4217-901a-6d0d06442cf4-trusted-ca\") pod \"console-operator-68b757fb67-k8dnq\" (UID: \"633a3ae5-3d78-4217-901a-6d0d06442cf4\") " pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" Jul 09 04:11:18.998736 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.998675 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-626bq\" (UniqueName: \"kubernetes.io/projected/633a3ae5-3d78-4217-901a-6d0d06442cf4-kube-api-access-626bq\") pod \"console-operator-68b757fb67-k8dnq\" (UID: \"633a3ae5-3d78-4217-901a-6d0d06442cf4\") " pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" Jul 09 04:11:18.998736 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:18.998713 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8872a475-7573-4057-a22c-236c5e337a1a-config\") pod \"service-ca-operator-5d46d9ffd4-w48tg\" (UID: \"8872a475-7573-4057-a22c-236c5e337a1a\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w48tg" Jul 09 04:11:19.011519 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.011484 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-2lxfd"] Jul 09 04:11:19.013993 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.013967 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-2lxfd" Jul 09 04:11:19.015459 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.015441 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9w76h"] Jul 09 04:11:19.016401 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.016383 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 09 04:11:19.016485 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.016466 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-jv7jr\"" Jul 09 04:11:19.016594 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.016579 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 09 04:11:19.016798 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.016782 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 09 04:11:19.016998 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.016984 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 09 04:11:19.017602 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.017581 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-p9f98"] Jul 09 04:11:19.017710 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.017697 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9w76h" Jul 09 04:11:19.019648 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.019630 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-p9f98" Jul 09 04:11:19.019967 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.019909 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 09 04:11:19.020113 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.020099 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 09 04:11:19.020185 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.020133 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-7998w\"" Jul 09 04:11:19.020413 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.020397 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 09 04:11:19.022108 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.022087 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-w462r\"" Jul 09 04:11:19.022483 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.022464 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 09 04:11:19.022577 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.022492 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 09 04:11:19.023590 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.023570 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-2lxfd"] Jul 09 04:11:19.028831 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.028807 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-p9f98"] Jul 09 04:11:19.030101 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.030082 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9w76h"] Jul 09 04:11:19.099965 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.099926 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/633a3ae5-3d78-4217-901a-6d0d06442cf4-trusted-ca\") pod \"console-operator-68b757fb67-k8dnq\" (UID: \"633a3ae5-3d78-4217-901a-6d0d06442cf4\") " pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" Jul 09 04:11:19.099965 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.099966 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-626bq\" (UniqueName: \"kubernetes.io/projected/633a3ae5-3d78-4217-901a-6d0d06442cf4-kube-api-access-626bq\") pod \"console-operator-68b757fb67-k8dnq\" (UID: \"633a3ae5-3d78-4217-901a-6d0d06442cf4\") " pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" Jul 09 04:11:19.100176 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.099990 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8872a475-7573-4057-a22c-236c5e337a1a-config\") pod \"service-ca-operator-5d46d9ffd4-w48tg\" (UID: \"8872a475-7573-4057-a22c-236c5e337a1a\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w48tg" Jul 09 04:11:19.100176 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.100020 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddzz9\" (UniqueName: \"kubernetes.io/projected/14521e76-b432-4e9f-95ee-9cb52fa8767c-kube-api-access-ddzz9\") pod \"volume-data-source-validator-5d456fdbcd-p9f98\" (UID: \"14521e76-b432-4e9f-95ee-9cb52fa8767c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-p9f98" Jul 09 04:11:19.100243 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.100168 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/900fd2e6-681b-451f-8f39-92d8d13bf9e5-serving-cert\") pod \"insights-operator-5c847dcfc4-8qmfm\" (UID: \"900fd2e6-681b-451f-8f39-92d8d13bf9e5\") " pod="openshift-insights/insights-operator-5c847dcfc4-8qmfm" Jul 09 04:11:19.100243 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.100209 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ad788a33-5689-40d9-89d1-5200f27e9549-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9w76h\" (UID: \"ad788a33-5689-40d9-89d1-5200f27e9549\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9w76h" Jul 09 04:11:19.100379 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.100240 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8872a475-7573-4057-a22c-236c5e337a1a-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-w48tg\" (UID: \"8872a475-7573-4057-a22c-236c5e337a1a\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w48tg" Jul 09 04:11:19.100379 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.100268 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/900fd2e6-681b-451f-8f39-92d8d13bf9e5-tmp\") pod \"insights-operator-5c847dcfc4-8qmfm\" (UID: \"900fd2e6-681b-451f-8f39-92d8d13bf9e5\") " pod="openshift-insights/insights-operator-5c847dcfc4-8qmfm" Jul 09 04:11:19.100469 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.100435 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49szk\" (UniqueName: \"kubernetes.io/projected/ad788a33-5689-40d9-89d1-5200f27e9549-kube-api-access-49szk\") pod \"cluster-samples-operator-56fcf86bf-9w76h\" (UID: \"ad788a33-5689-40d9-89d1-5200f27e9549\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9w76h" Jul 09 04:11:19.100526 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.100510 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/900fd2e6-681b-451f-8f39-92d8d13bf9e5-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-8qmfm\" (UID: \"900fd2e6-681b-451f-8f39-92d8d13bf9e5\") " pod="openshift-insights/insights-operator-5c847dcfc4-8qmfm" Jul 09 04:11:19.100586 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.100542 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hchp5\" (UniqueName: \"kubernetes.io/projected/900fd2e6-681b-451f-8f39-92d8d13bf9e5-kube-api-access-hchp5\") pod \"insights-operator-5c847dcfc4-8qmfm\" (UID: \"900fd2e6-681b-451f-8f39-92d8d13bf9e5\") " pod="openshift-insights/insights-operator-5c847dcfc4-8qmfm" Jul 09 04:11:19.100586 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.100572 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/900fd2e6-681b-451f-8f39-92d8d13bf9e5-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-8qmfm\" (UID: \"900fd2e6-681b-451f-8f39-92d8d13bf9e5\") " pod="openshift-insights/insights-operator-5c847dcfc4-8qmfm" Jul 09 04:11:19.100680 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.100598 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsqn9\" (UniqueName: \"kubernetes.io/projected/8872a475-7573-4057-a22c-236c5e337a1a-kube-api-access-hsqn9\") pod \"service-ca-operator-5d46d9ffd4-w48tg\" (UID: \"8872a475-7573-4057-a22c-236c5e337a1a\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w48tg" Jul 09 04:11:19.100680 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.100628 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjvvp\" (UniqueName: \"kubernetes.io/projected/0c2522b9-d54d-4091-907e-eb613d77b5c9-kube-api-access-gjvvp\") pod \"kube-storage-version-migrator-operator-846bf668f7-2lxfd\" (UID: \"0c2522b9-d54d-4091-907e-eb613d77b5c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-2lxfd" Jul 09 04:11:19.100680 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.100657 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/900fd2e6-681b-451f-8f39-92d8d13bf9e5-tmp\") pod \"insights-operator-5c847dcfc4-8qmfm\" (UID: \"900fd2e6-681b-451f-8f39-92d8d13bf9e5\") " pod="openshift-insights/insights-operator-5c847dcfc4-8qmfm" Jul 09 04:11:19.100680 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.100671 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0c2522b9-d54d-4091-907e-eb613d77b5c9-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-2lxfd\" (UID: \"0c2522b9-d54d-4091-907e-eb613d77b5c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-2lxfd" Jul 09 04:11:19.100886 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.100725 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/633a3ae5-3d78-4217-901a-6d0d06442cf4-serving-cert\") pod \"console-operator-68b757fb67-k8dnq\" (UID: \"633a3ae5-3d78-4217-901a-6d0d06442cf4\") " pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" Jul 09 04:11:19.100886 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.100731 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8872a475-7573-4057-a22c-236c5e337a1a-config\") pod \"service-ca-operator-5d46d9ffd4-w48tg\" (UID: \"8872a475-7573-4057-a22c-236c5e337a1a\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w48tg" Jul 09 04:11:19.100886 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.100783 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0c2522b9-d54d-4091-907e-eb613d77b5c9-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-2lxfd\" (UID: \"0c2522b9-d54d-4091-907e-eb613d77b5c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-2lxfd" Jul 09 04:11:19.100886 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.100822 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/633a3ae5-3d78-4217-901a-6d0d06442cf4-config\") pod \"console-operator-68b757fb67-k8dnq\" (UID: \"633a3ae5-3d78-4217-901a-6d0d06442cf4\") " pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" Jul 09 04:11:19.100886 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.100847 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/900fd2e6-681b-451f-8f39-92d8d13bf9e5-snapshots\") pod \"insights-operator-5c847dcfc4-8qmfm\" (UID: \"900fd2e6-681b-451f-8f39-92d8d13bf9e5\") " pod="openshift-insights/insights-operator-5c847dcfc4-8qmfm" Jul 09 04:11:19.101303 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.101086 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/900fd2e6-681b-451f-8f39-92d8d13bf9e5-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-8qmfm\" (UID: \"900fd2e6-681b-451f-8f39-92d8d13bf9e5\") " pod="openshift-insights/insights-operator-5c847dcfc4-8qmfm" Jul 09 04:11:19.101401 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.101375 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/633a3ae5-3d78-4217-901a-6d0d06442cf4-trusted-ca\") pod \"console-operator-68b757fb67-k8dnq\" (UID: \"633a3ae5-3d78-4217-901a-6d0d06442cf4\") " pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" Jul 09 04:11:19.101568 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.101545 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/900fd2e6-681b-451f-8f39-92d8d13bf9e5-snapshots\") pod \"insights-operator-5c847dcfc4-8qmfm\" (UID: \"900fd2e6-681b-451f-8f39-92d8d13bf9e5\") " pod="openshift-insights/insights-operator-5c847dcfc4-8qmfm" Jul 09 04:11:19.101650 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.101580 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/900fd2e6-681b-451f-8f39-92d8d13bf9e5-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-8qmfm\" (UID: \"900fd2e6-681b-451f-8f39-92d8d13bf9e5\") " pod="openshift-insights/insights-operator-5c847dcfc4-8qmfm" Jul 09 04:11:19.101740 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.101721 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/633a3ae5-3d78-4217-901a-6d0d06442cf4-config\") pod \"console-operator-68b757fb67-k8dnq\" (UID: \"633a3ae5-3d78-4217-901a-6d0d06442cf4\") " pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" Jul 09 04:11:19.102778 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.102755 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8872a475-7573-4057-a22c-236c5e337a1a-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-w48tg\" (UID: \"8872a475-7573-4057-a22c-236c5e337a1a\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w48tg" Jul 09 04:11:19.103251 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.103235 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/900fd2e6-681b-451f-8f39-92d8d13bf9e5-serving-cert\") pod \"insights-operator-5c847dcfc4-8qmfm\" (UID: \"900fd2e6-681b-451f-8f39-92d8d13bf9e5\") " pod="openshift-insights/insights-operator-5c847dcfc4-8qmfm" Jul 09 04:11:19.103470 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.103456 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/633a3ae5-3d78-4217-901a-6d0d06442cf4-serving-cert\") pod \"console-operator-68b757fb67-k8dnq\" (UID: \"633a3ae5-3d78-4217-901a-6d0d06442cf4\") " pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" Jul 09 04:11:19.109398 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.109380 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsqn9\" (UniqueName: \"kubernetes.io/projected/8872a475-7573-4057-a22c-236c5e337a1a-kube-api-access-hsqn9\") pod \"service-ca-operator-5d46d9ffd4-w48tg\" (UID: \"8872a475-7573-4057-a22c-236c5e337a1a\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w48tg" Jul 09 04:11:19.109938 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.109920 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hchp5\" (UniqueName: \"kubernetes.io/projected/900fd2e6-681b-451f-8f39-92d8d13bf9e5-kube-api-access-hchp5\") pod \"insights-operator-5c847dcfc4-8qmfm\" (UID: \"900fd2e6-681b-451f-8f39-92d8d13bf9e5\") " pod="openshift-insights/insights-operator-5c847dcfc4-8qmfm" Jul 09 04:11:19.110143 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.110124 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-626bq\" (UniqueName: \"kubernetes.io/projected/633a3ae5-3d78-4217-901a-6d0d06442cf4-kube-api-access-626bq\") pod \"console-operator-68b757fb67-k8dnq\" (UID: \"633a3ae5-3d78-4217-901a-6d0d06442cf4\") " pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" Jul 09 04:11:19.201933 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.201837 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddzz9\" (UniqueName: \"kubernetes.io/projected/14521e76-b432-4e9f-95ee-9cb52fa8767c-kube-api-access-ddzz9\") pod \"volume-data-source-validator-5d456fdbcd-p9f98\" (UID: \"14521e76-b432-4e9f-95ee-9cb52fa8767c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-p9f98" Jul 09 04:11:19.201933 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.201890 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ad788a33-5689-40d9-89d1-5200f27e9549-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9w76h\" (UID: \"ad788a33-5689-40d9-89d1-5200f27e9549\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9w76h" Jul 09 04:11:19.201933 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.201908 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49szk\" (UniqueName: \"kubernetes.io/projected/ad788a33-5689-40d9-89d1-5200f27e9549-kube-api-access-49szk\") pod \"cluster-samples-operator-56fcf86bf-9w76h\" (UID: \"ad788a33-5689-40d9-89d1-5200f27e9549\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9w76h" Jul 09 04:11:19.202269 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:19.202000 2534 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 04:11:19.202269 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.202063 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjvvp\" (UniqueName: \"kubernetes.io/projected/0c2522b9-d54d-4091-907e-eb613d77b5c9-kube-api-access-gjvvp\") pod \"kube-storage-version-migrator-operator-846bf668f7-2lxfd\" (UID: \"0c2522b9-d54d-4091-907e-eb613d77b5c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-2lxfd" Jul 09 04:11:19.202269 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:19.202076 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad788a33-5689-40d9-89d1-5200f27e9549-samples-operator-tls podName:ad788a33-5689-40d9-89d1-5200f27e9549 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:19.702057761 +0000 UTC m=+269.925676229 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ad788a33-5689-40d9-89d1-5200f27e9549-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-9w76h" (UID: "ad788a33-5689-40d9-89d1-5200f27e9549") : secret "samples-operator-tls" not found Jul 09 04:11:19.202269 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.202113 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0c2522b9-d54d-4091-907e-eb613d77b5c9-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-2lxfd\" (UID: \"0c2522b9-d54d-4091-907e-eb613d77b5c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-2lxfd" Jul 09 04:11:19.202269 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.202165 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0c2522b9-d54d-4091-907e-eb613d77b5c9-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-2lxfd\" (UID: \"0c2522b9-d54d-4091-907e-eb613d77b5c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-2lxfd" Jul 09 04:11:19.202740 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.202723 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0c2522b9-d54d-4091-907e-eb613d77b5c9-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-2lxfd\" (UID: \"0c2522b9-d54d-4091-907e-eb613d77b5c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-2lxfd" Jul 09 04:11:19.204497 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.204480 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0c2522b9-d54d-4091-907e-eb613d77b5c9-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-2lxfd\" (UID: \"0c2522b9-d54d-4091-907e-eb613d77b5c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-2lxfd" Jul 09 04:11:19.210073 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.210050 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjvvp\" (UniqueName: \"kubernetes.io/projected/0c2522b9-d54d-4091-907e-eb613d77b5c9-kube-api-access-gjvvp\") pod \"kube-storage-version-migrator-operator-846bf668f7-2lxfd\" (UID: \"0c2522b9-d54d-4091-907e-eb613d77b5c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-2lxfd" Jul 09 04:11:19.210846 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.210829 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49szk\" (UniqueName: \"kubernetes.io/projected/ad788a33-5689-40d9-89d1-5200f27e9549-kube-api-access-49szk\") pod \"cluster-samples-operator-56fcf86bf-9w76h\" (UID: \"ad788a33-5689-40d9-89d1-5200f27e9549\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9w76h" Jul 09 04:11:19.211554 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.211537 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddzz9\" (UniqueName: \"kubernetes.io/projected/14521e76-b432-4e9f-95ee-9cb52fa8767c-kube-api-access-ddzz9\") pod \"volume-data-source-validator-5d456fdbcd-p9f98\" (UID: \"14521e76-b432-4e9f-95ee-9cb52fa8767c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-p9f98" Jul 09 04:11:19.231372 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.231339 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w48tg" Jul 09 04:11:19.239958 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.239928 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-8qmfm" Jul 09 04:11:19.246125 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.246106 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" Jul 09 04:11:19.324701 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.324660 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-2lxfd" Jul 09 04:11:19.338878 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.338562 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-p9f98" Jul 09 04:11:19.379162 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.379130 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w48tg"] Jul 09 04:11:19.383121 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:11:19.383088 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8872a475_7573_4057_a22c_236c5e337a1a.slice/crio-834bc1c21ef374bc0a8b42abff664f5a4e570974ff89d3380e50e6ba2ba725f9 WatchSource:0}: Error finding container 834bc1c21ef374bc0a8b42abff664f5a4e570974ff89d3380e50e6ba2ba725f9: Status 404 returned error can't find the container with id 834bc1c21ef374bc0a8b42abff664f5a4e570974ff89d3380e50e6ba2ba725f9 Jul 09 04:11:19.397118 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.397051 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-k8dnq"] Jul 09 04:11:19.404973 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:11:19.404938 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod633a3ae5_3d78_4217_901a_6d0d06442cf4.slice/crio-f2d56a1d055324241648f2d8f5a22c7af3346f6a2dafbfa12524854f98dbb44e WatchSource:0}: Error finding container f2d56a1d055324241648f2d8f5a22c7af3346f6a2dafbfa12524854f98dbb44e: Status 404 returned error can't find the container with id f2d56a1d055324241648f2d8f5a22c7af3346f6a2dafbfa12524854f98dbb44e Jul 09 04:11:19.490557 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.490524 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-p9f98"] Jul 09 04:11:19.493448 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:11:19.493421 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14521e76_b432_4e9f_95ee_9cb52fa8767c.slice/crio-02795c7ec744802bbe6db2c9c866c48f32bbf4790fc45ee611c72cf6a4bd703e WatchSource:0}: Error finding container 02795c7ec744802bbe6db2c9c866c48f32bbf4790fc45ee611c72cf6a4bd703e: Status 404 returned error can't find the container with id 02795c7ec744802bbe6db2c9c866c48f32bbf4790fc45ee611c72cf6a4bd703e Jul 09 04:11:19.507757 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.507732 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-2lxfd"] Jul 09 04:11:19.511080 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:11:19.511048 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c2522b9_d54d_4091_907e_eb613d77b5c9.slice/crio-980f36ad946ef05f26d4325ae05e0a580456ede5dc9ccb1bb6789f6f15fe2ba7 WatchSource:0}: Error finding container 980f36ad946ef05f26d4325ae05e0a580456ede5dc9ccb1bb6789f6f15fe2ba7: Status 404 returned error can't find the container with id 980f36ad946ef05f26d4325ae05e0a580456ede5dc9ccb1bb6789f6f15fe2ba7 Jul 09 04:11:19.599554 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.599522 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-8qmfm"] Jul 09 04:11:19.602894 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:11:19.602870 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod900fd2e6_681b_451f_8f39_92d8d13bf9e5.slice/crio-d92da9492350675054881fb9603b0df3b90c6748fcce12a0fa5ec424060d8f8a WatchSource:0}: Error finding container d92da9492350675054881fb9603b0df3b90c6748fcce12a0fa5ec424060d8f8a: Status 404 returned error can't find the container with id d92da9492350675054881fb9603b0df3b90c6748fcce12a0fa5ec424060d8f8a Jul 09 04:11:19.707972 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:19.707934 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ad788a33-5689-40d9-89d1-5200f27e9549-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9w76h\" (UID: \"ad788a33-5689-40d9-89d1-5200f27e9549\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9w76h" Jul 09 04:11:19.708132 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:19.708087 2534 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 04:11:19.708182 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:19.708150 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad788a33-5689-40d9-89d1-5200f27e9549-samples-operator-tls podName:ad788a33-5689-40d9-89d1-5200f27e9549 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:20.70813316 +0000 UTC m=+270.931751647 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ad788a33-5689-40d9-89d1-5200f27e9549-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-9w76h" (UID: "ad788a33-5689-40d9-89d1-5200f27e9549") : secret "samples-operator-tls" not found Jul 09 04:11:20.033244 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:20.033205 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" event={"ID":"633a3ae5-3d78-4217-901a-6d0d06442cf4","Type":"ContainerStarted","Data":"f2d56a1d055324241648f2d8f5a22c7af3346f6a2dafbfa12524854f98dbb44e"} Jul 09 04:11:20.034292 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:20.034252 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w48tg" event={"ID":"8872a475-7573-4057-a22c-236c5e337a1a","Type":"ContainerStarted","Data":"834bc1c21ef374bc0a8b42abff664f5a4e570974ff89d3380e50e6ba2ba725f9"} Jul 09 04:11:20.035223 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:20.035198 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-8qmfm" event={"ID":"900fd2e6-681b-451f-8f39-92d8d13bf9e5","Type":"ContainerStarted","Data":"d92da9492350675054881fb9603b0df3b90c6748fcce12a0fa5ec424060d8f8a"} Jul 09 04:11:20.036194 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:20.036173 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-p9f98" event={"ID":"14521e76-b432-4e9f-95ee-9cb52fa8767c","Type":"ContainerStarted","Data":"02795c7ec744802bbe6db2c9c866c48f32bbf4790fc45ee611c72cf6a4bd703e"} Jul 09 04:11:20.037167 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:20.037148 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-2lxfd" event={"ID":"0c2522b9-d54d-4091-907e-eb613d77b5c9","Type":"ContainerStarted","Data":"980f36ad946ef05f26d4325ae05e0a580456ede5dc9ccb1bb6789f6f15fe2ba7"} Jul 09 04:11:20.718189 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:20.718148 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ad788a33-5689-40d9-89d1-5200f27e9549-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9w76h\" (UID: \"ad788a33-5689-40d9-89d1-5200f27e9549\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9w76h" Jul 09 04:11:20.718389 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:20.718342 2534 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 04:11:20.718461 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:20.718432 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad788a33-5689-40d9-89d1-5200f27e9549-samples-operator-tls podName:ad788a33-5689-40d9-89d1-5200f27e9549 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:22.718410591 +0000 UTC m=+272.942029058 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ad788a33-5689-40d9-89d1-5200f27e9549-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-9w76h" (UID: "ad788a33-5689-40d9-89d1-5200f27e9549") : secret "samples-operator-tls" not found Jul 09 04:11:22.045359 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:22.045326 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-p9f98" event={"ID":"14521e76-b432-4e9f-95ee-9cb52fa8767c","Type":"ContainerStarted","Data":"a798dc6191f00590602ad35515dd7035eca117145bffd1673a34c66e35509482"} Jul 09 04:11:22.059717 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:22.059672 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-p9f98" podStartSLOduration=2.5724731739999998 podStartE2EDuration="4.059657576s" totalImagesPullingTime="1.487184402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:11:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:11:22.058941754 +0000 UTC m=+272.282560243" watchObservedRunningTime="2026-07-09 04:11:22.059657576 +0000 UTC m=+272.283276063" Jul 09 04:11:22.737452 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:22.737413 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ad788a33-5689-40d9-89d1-5200f27e9549-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9w76h\" (UID: \"ad788a33-5689-40d9-89d1-5200f27e9549\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9w76h" Jul 09 04:11:22.737620 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:22.737562 2534 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 04:11:22.737684 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:22.737630 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad788a33-5689-40d9-89d1-5200f27e9549-samples-operator-tls podName:ad788a33-5689-40d9-89d1-5200f27e9549 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:26.737615014 +0000 UTC m=+276.961233480 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ad788a33-5689-40d9-89d1-5200f27e9549-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-9w76h" (UID: "ad788a33-5689-40d9-89d1-5200f27e9549") : secret "samples-operator-tls" not found Jul 09 04:11:24.053305 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:24.053245 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w48tg" event={"ID":"8872a475-7573-4057-a22c-236c5e337a1a","Type":"ContainerStarted","Data":"b181615c6acf5f89003ba97cc61dbf98d4b04996fdea725590d9ca958a270cc6"} Jul 09 04:11:24.055359 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:24.055319 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-8qmfm" event={"ID":"900fd2e6-681b-451f-8f39-92d8d13bf9e5","Type":"ContainerStarted","Data":"a2168fc959b068da8a0ca38813fddd5c8f5a5cb231d956e1cfc3c87e6bd08fae"} Jul 09 04:11:24.056947 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:24.056901 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-2lxfd" event={"ID":"0c2522b9-d54d-4091-907e-eb613d77b5c9","Type":"ContainerStarted","Data":"c96c66e1a0fa7fde51ddbb4a8bef32367c766abac052be4ab87d969c0bb82611"} Jul 09 04:11:24.058649 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:24.058630 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8dnq_633a3ae5-3d78-4217-901a-6d0d06442cf4/console-operator/0.log" Jul 09 04:11:24.058752 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:24.058668 2534 generic.go:356] "Generic (PLEG): container finished" podID="633a3ae5-3d78-4217-901a-6d0d06442cf4" containerID="e575406a91fb6d61fb6b37d3f338997cf7c89dafe74d2e49a0ec94ffbe504ee7" exitCode=255 Jul 09 04:11:24.058752 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:24.058696 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" event={"ID":"633a3ae5-3d78-4217-901a-6d0d06442cf4","Type":"ContainerDied","Data":"e575406a91fb6d61fb6b37d3f338997cf7c89dafe74d2e49a0ec94ffbe504ee7"} Jul 09 04:11:24.058899 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:24.058884 2534 scope.go:117] "RemoveContainer" containerID="e575406a91fb6d61fb6b37d3f338997cf7c89dafe74d2e49a0ec94ffbe504ee7" Jul 09 04:11:24.069118 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:24.069073 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w48tg" podStartSLOduration=1.938023686 podStartE2EDuration="6.069058709s" totalImagesPullingTime="4.131035023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:11:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:11:24.067883765 +0000 UTC m=+274.291502253" watchObservedRunningTime="2026-07-09 04:11:24.069058709 +0000 UTC m=+274.292677198" Jul 09 04:11:24.082823 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:24.082771 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-2lxfd" podStartSLOduration=2.085563695 podStartE2EDuration="6.082749407s" totalImagesPullingTime="3.997185712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:11:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:11:24.082379912 +0000 UTC m=+274.305998409" watchObservedRunningTime="2026-07-09 04:11:24.082749407 +0000 UTC m=+274.306367896" Jul 09 04:11:24.101117 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:24.101064 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-8qmfm" podStartSLOduration=2.192534554 podStartE2EDuration="6.101044014s" totalImagesPullingTime="3.90850946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:11:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:11:24.10048471 +0000 UTC m=+274.324103201" watchObservedRunningTime="2026-07-09 04:11:24.101044014 +0000 UTC m=+274.324662502" Jul 09 04:11:24.886308 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:24.885789 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9hgjm"] Jul 09 04:11:24.888484 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:24.888457 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:24.890929 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:24.890875 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 04:11:24.891440 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:24.891116 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 04:11:24.891440 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:24.891229 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-c6c2f\"" Jul 09 04:11:24.907296 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:24.907252 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9hgjm"] Jul 09 04:11:25.058716 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.058684 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/85856d16-c1e7-4990-9982-6e01d8ebc7f6-data-volume\") pod \"insights-runtime-extractor-9hgjm\" (UID: \"85856d16-c1e7-4990-9982-6e01d8ebc7f6\") " pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:25.059134 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.058762 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/85856d16-c1e7-4990-9982-6e01d8ebc7f6-crio-socket\") pod \"insights-runtime-extractor-9hgjm\" (UID: \"85856d16-c1e7-4990-9982-6e01d8ebc7f6\") " pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:25.059134 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.058797 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/85856d16-c1e7-4990-9982-6e01d8ebc7f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9hgjm\" (UID: \"85856d16-c1e7-4990-9982-6e01d8ebc7f6\") " pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:25.059134 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.058883 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/85856d16-c1e7-4990-9982-6e01d8ebc7f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9hgjm\" (UID: \"85856d16-c1e7-4990-9982-6e01d8ebc7f6\") " pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:25.059134 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.058928 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5rln\" (UniqueName: \"kubernetes.io/projected/85856d16-c1e7-4990-9982-6e01d8ebc7f6-kube-api-access-x5rln\") pod \"insights-runtime-extractor-9hgjm\" (UID: \"85856d16-c1e7-4990-9982-6e01d8ebc7f6\") " pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:25.063484 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.063462 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8dnq_633a3ae5-3d78-4217-901a-6d0d06442cf4/console-operator/1.log" Jul 09 04:11:25.063905 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.063892 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8dnq_633a3ae5-3d78-4217-901a-6d0d06442cf4/console-operator/0.log" Jul 09 04:11:25.063974 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.063923 2534 generic.go:356] "Generic (PLEG): container finished" podID="633a3ae5-3d78-4217-901a-6d0d06442cf4" containerID="b7ff8a77bac5aa0a21e538caf32b72ad0d9e20ffa2150627acc29f534ffa168b" exitCode=255 Jul 09 04:11:25.063974 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.063952 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" event={"ID":"633a3ae5-3d78-4217-901a-6d0d06442cf4","Type":"ContainerDied","Data":"b7ff8a77bac5aa0a21e538caf32b72ad0d9e20ffa2150627acc29f534ffa168b"} Jul 09 04:11:25.064060 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.064019 2534 scope.go:117] "RemoveContainer" containerID="e575406a91fb6d61fb6b37d3f338997cf7c89dafe74d2e49a0ec94ffbe504ee7" Jul 09 04:11:25.064241 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.064218 2534 scope.go:117] "RemoveContainer" containerID="b7ff8a77bac5aa0a21e538caf32b72ad0d9e20ffa2150627acc29f534ffa168b" Jul 09 04:11:25.064464 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:25.064434 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-k8dnq_openshift-console-operator(633a3ae5-3d78-4217-901a-6d0d06442cf4)\"" pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" podUID="633a3ae5-3d78-4217-901a-6d0d06442cf4" Jul 09 04:11:25.160194 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.160110 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/85856d16-c1e7-4990-9982-6e01d8ebc7f6-data-volume\") pod \"insights-runtime-extractor-9hgjm\" (UID: \"85856d16-c1e7-4990-9982-6e01d8ebc7f6\") " pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:25.160423 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.160398 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/85856d16-c1e7-4990-9982-6e01d8ebc7f6-crio-socket\") pod \"insights-runtime-extractor-9hgjm\" (UID: \"85856d16-c1e7-4990-9982-6e01d8ebc7f6\") " pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:25.160523 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.160504 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/85856d16-c1e7-4990-9982-6e01d8ebc7f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9hgjm\" (UID: \"85856d16-c1e7-4990-9982-6e01d8ebc7f6\") " pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:25.160583 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.160512 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/85856d16-c1e7-4990-9982-6e01d8ebc7f6-crio-socket\") pod \"insights-runtime-extractor-9hgjm\" (UID: \"85856d16-c1e7-4990-9982-6e01d8ebc7f6\") " pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:25.160636 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:25.160609 2534 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 04:11:25.160692 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:25.160667 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85856d16-c1e7-4990-9982-6e01d8ebc7f6-insights-runtime-extractor-tls podName:85856d16-c1e7-4990-9982-6e01d8ebc7f6 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:25.660647306 +0000 UTC m=+275.884265790 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/85856d16-c1e7-4990-9982-6e01d8ebc7f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9hgjm" (UID: "85856d16-c1e7-4990-9982-6e01d8ebc7f6") : secret "insights-runtime-extractor-tls" not found Jul 09 04:11:25.160831 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.160804 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/85856d16-c1e7-4990-9982-6e01d8ebc7f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9hgjm\" (UID: \"85856d16-c1e7-4990-9982-6e01d8ebc7f6\") " pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:25.160936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.160915 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5rln\" (UniqueName: \"kubernetes.io/projected/85856d16-c1e7-4990-9982-6e01d8ebc7f6-kube-api-access-x5rln\") pod \"insights-runtime-extractor-9hgjm\" (UID: \"85856d16-c1e7-4990-9982-6e01d8ebc7f6\") " pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:25.160993 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:25.160964 2534 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 04:11:25.161047 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:25.161029 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/85856d16-c1e7-4990-9982-6e01d8ebc7f6-kube-rbac-proxy-cm podName:85856d16-c1e7-4990-9982-6e01d8ebc7f6 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:25.661011389 +0000 UTC m=+275.884629870 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/85856d16-c1e7-4990-9982-6e01d8ebc7f6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9hgjm" (UID: "85856d16-c1e7-4990-9982-6e01d8ebc7f6") : configmap "kube-rbac-proxy" not found Jul 09 04:11:25.161177 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.161156 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/85856d16-c1e7-4990-9982-6e01d8ebc7f6-data-volume\") pod \"insights-runtime-extractor-9hgjm\" (UID: \"85856d16-c1e7-4990-9982-6e01d8ebc7f6\") " pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:25.168682 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.168655 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5rln\" (UniqueName: \"kubernetes.io/projected/85856d16-c1e7-4990-9982-6e01d8ebc7f6-kube-api-access-x5rln\") pod \"insights-runtime-extractor-9hgjm\" (UID: \"85856d16-c1e7-4990-9982-6e01d8ebc7f6\") " pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:25.406913 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.406870 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-rkghl"] Jul 09 04:11:25.410074 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.410046 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-rkghl" Jul 09 04:11:25.412569 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.412505 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 09 04:11:25.412687 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.412575 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 09 04:11:25.412957 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.412939 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-nn8bk\"" Jul 09 04:11:25.421940 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.421906 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-rkghl"] Jul 09 04:11:25.564724 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.564690 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7x7b\" (UniqueName: \"kubernetes.io/projected/195e8f20-175e-4e62-a0f8-1635f6d29111-kube-api-access-j7x7b\") pod \"migrator-d57d58cc9-rkghl\" (UID: \"195e8f20-175e-4e62-a0f8-1635f6d29111\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-rkghl" Jul 09 04:11:25.665346 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.665252 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/85856d16-c1e7-4990-9982-6e01d8ebc7f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9hgjm\" (UID: \"85856d16-c1e7-4990-9982-6e01d8ebc7f6\") " pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:25.665346 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.665320 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7x7b\" (UniqueName: \"kubernetes.io/projected/195e8f20-175e-4e62-a0f8-1635f6d29111-kube-api-access-j7x7b\") pod \"migrator-d57d58cc9-rkghl\" (UID: \"195e8f20-175e-4e62-a0f8-1635f6d29111\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-rkghl" Jul 09 04:11:25.665346 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.665342 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/85856d16-c1e7-4990-9982-6e01d8ebc7f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9hgjm\" (UID: \"85856d16-c1e7-4990-9982-6e01d8ebc7f6\") " pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:25.665544 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:25.665411 2534 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 04:11:25.665544 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:25.665427 2534 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 04:11:25.665544 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:25.665487 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85856d16-c1e7-4990-9982-6e01d8ebc7f6-insights-runtime-extractor-tls podName:85856d16-c1e7-4990-9982-6e01d8ebc7f6 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:26.665467313 +0000 UTC m=+276.889085793 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/85856d16-c1e7-4990-9982-6e01d8ebc7f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9hgjm" (UID: "85856d16-c1e7-4990-9982-6e01d8ebc7f6") : secret "insights-runtime-extractor-tls" not found Jul 09 04:11:25.665544 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:25.665508 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/85856d16-c1e7-4990-9982-6e01d8ebc7f6-kube-rbac-proxy-cm podName:85856d16-c1e7-4990-9982-6e01d8ebc7f6 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:26.665497603 +0000 UTC m=+276.889116069 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/85856d16-c1e7-4990-9982-6e01d8ebc7f6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9hgjm" (UID: "85856d16-c1e7-4990-9982-6e01d8ebc7f6") : configmap "kube-rbac-proxy" not found Jul 09 04:11:25.672749 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.672728 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7x7b\" (UniqueName: \"kubernetes.io/projected/195e8f20-175e-4e62-a0f8-1635f6d29111-kube-api-access-j7x7b\") pod \"migrator-d57d58cc9-rkghl\" (UID: \"195e8f20-175e-4e62-a0f8-1635f6d29111\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-rkghl" Jul 09 04:11:25.726533 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.726510 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-rkghl" Jul 09 04:11:25.846677 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:25.846642 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-rkghl"] Jul 09 04:11:25.849370 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:11:25.849345 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod195e8f20_175e_4e62_a0f8_1635f6d29111.slice/crio-f1d05d44fe14a542c84b789c2d52f02ca52163d13e66ec9a0c96324eda6a9c1c WatchSource:0}: Error finding container f1d05d44fe14a542c84b789c2d52f02ca52163d13e66ec9a0c96324eda6a9c1c: Status 404 returned error can't find the container with id f1d05d44fe14a542c84b789c2d52f02ca52163d13e66ec9a0c96324eda6a9c1c Jul 09 04:11:26.068039 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:26.068005 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-rkghl" event={"ID":"195e8f20-175e-4e62-a0f8-1635f6d29111","Type":"ContainerStarted","Data":"f1d05d44fe14a542c84b789c2d52f02ca52163d13e66ec9a0c96324eda6a9c1c"} Jul 09 04:11:26.069461 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:26.069444 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8dnq_633a3ae5-3d78-4217-901a-6d0d06442cf4/console-operator/1.log" Jul 09 04:11:26.069759 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:26.069745 2534 scope.go:117] "RemoveContainer" containerID="b7ff8a77bac5aa0a21e538caf32b72ad0d9e20ffa2150627acc29f534ffa168b" Jul 09 04:11:26.069899 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:26.069883 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-k8dnq_openshift-console-operator(633a3ae5-3d78-4217-901a-6d0d06442cf4)\"" pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" podUID="633a3ae5-3d78-4217-901a-6d0d06442cf4" Jul 09 04:11:26.674436 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:26.674405 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/85856d16-c1e7-4990-9982-6e01d8ebc7f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9hgjm\" (UID: \"85856d16-c1e7-4990-9982-6e01d8ebc7f6\") " pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:26.674618 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:26.674462 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/85856d16-c1e7-4990-9982-6e01d8ebc7f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9hgjm\" (UID: \"85856d16-c1e7-4990-9982-6e01d8ebc7f6\") " pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:26.674618 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:26.674557 2534 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 04:11:26.674618 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:26.674615 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85856d16-c1e7-4990-9982-6e01d8ebc7f6-insights-runtime-extractor-tls podName:85856d16-c1e7-4990-9982-6e01d8ebc7f6 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:28.674601129 +0000 UTC m=+278.898219594 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/85856d16-c1e7-4990-9982-6e01d8ebc7f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9hgjm" (UID: "85856d16-c1e7-4990-9982-6e01d8ebc7f6") : secret "insights-runtime-extractor-tls" not found Jul 09 04:11:26.674766 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:26.674557 2534 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 04:11:26.674766 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:26.674675 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/85856d16-c1e7-4990-9982-6e01d8ebc7f6-kube-rbac-proxy-cm podName:85856d16-c1e7-4990-9982-6e01d8ebc7f6 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:28.674662753 +0000 UTC m=+278.898281221 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/85856d16-c1e7-4990-9982-6e01d8ebc7f6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9hgjm" (UID: "85856d16-c1e7-4990-9982-6e01d8ebc7f6") : configmap "kube-rbac-proxy" not found Jul 09 04:11:26.775684 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:26.775645 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ad788a33-5689-40d9-89d1-5200f27e9549-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9w76h\" (UID: \"ad788a33-5689-40d9-89d1-5200f27e9549\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9w76h" Jul 09 04:11:26.775883 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:26.775822 2534 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 04:11:26.775949 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:26.775902 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad788a33-5689-40d9-89d1-5200f27e9549-samples-operator-tls podName:ad788a33-5689-40d9-89d1-5200f27e9549 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:34.775878848 +0000 UTC m=+284.999497318 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ad788a33-5689-40d9-89d1-5200f27e9549-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-9w76h" (UID: "ad788a33-5689-40d9-89d1-5200f27e9549") : secret "samples-operator-tls" not found Jul 09 04:11:27.089431 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:27.089388 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zpr9q_8104c5b9-011c-4e10-9a15-53bdd52870f0/dns-node-resolver/0.log" Jul 09 04:11:27.888638 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:27.888612 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dgz8s_0a7d54c6-9262-4462-8b4b-f35a678e2ddf/node-ca/0.log" Jul 09 04:11:28.078295 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:28.078247 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-rkghl" event={"ID":"195e8f20-175e-4e62-a0f8-1635f6d29111","Type":"ContainerStarted","Data":"b21e42ea90c4933d0ced41935619ea134a7a3e5ce5f5ac356b149ba853754505"} Jul 09 04:11:28.078458 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:28.078312 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-rkghl" event={"ID":"195e8f20-175e-4e62-a0f8-1635f6d29111","Type":"ContainerStarted","Data":"5ef166276a4bae5e413f8b184b8fc0b0c59d188361bfe321a756f8835d7a80dd"} Jul 09 04:11:28.095351 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:28.095306 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-rkghl" podStartSLOduration=1.598896695 podStartE2EDuration="3.095271274s" totalImagesPullingTime="1.496374579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:11:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:11:28.093888006 +0000 UTC m=+278.317506494" watchObservedRunningTime="2026-07-09 04:11:28.095271274 +0000 UTC m=+278.318889762" Jul 09 04:11:28.690737 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:28.690704 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/85856d16-c1e7-4990-9982-6e01d8ebc7f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9hgjm\" (UID: \"85856d16-c1e7-4990-9982-6e01d8ebc7f6\") " pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:28.690940 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:28.690806 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/85856d16-c1e7-4990-9982-6e01d8ebc7f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9hgjm\" (UID: \"85856d16-c1e7-4990-9982-6e01d8ebc7f6\") " pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:28.690940 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:28.690855 2534 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 04:11:28.690940 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:28.690894 2534 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 04:11:28.690940 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:28.690937 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/85856d16-c1e7-4990-9982-6e01d8ebc7f6-kube-rbac-proxy-cm podName:85856d16-c1e7-4990-9982-6e01d8ebc7f6 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:32.69091734 +0000 UTC m=+282.914535822 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/85856d16-c1e7-4990-9982-6e01d8ebc7f6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9hgjm" (UID: "85856d16-c1e7-4990-9982-6e01d8ebc7f6") : configmap "kube-rbac-proxy" not found Jul 09 04:11:28.691169 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:28.690958 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85856d16-c1e7-4990-9982-6e01d8ebc7f6-insights-runtime-extractor-tls podName:85856d16-c1e7-4990-9982-6e01d8ebc7f6 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:32.690948524 +0000 UTC m=+282.914566997 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/85856d16-c1e7-4990-9982-6e01d8ebc7f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9hgjm" (UID: "85856d16-c1e7-4990-9982-6e01d8ebc7f6") : secret "insights-runtime-extractor-tls" not found Jul 09 04:11:28.889230 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:28.889201 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ktvhk_3eb622e3-4ce7-4fe0-a2ae-b5c83a81fde6/serve-healthcheck-canary/0.log" Jul 09 04:11:29.246264 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:29.246231 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" Jul 09 04:11:29.246655 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:29.246302 2534 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" Jul 09 04:11:29.246655 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:29.246593 2534 scope.go:117] "RemoveContainer" containerID="b7ff8a77bac5aa0a21e538caf32b72ad0d9e20ffa2150627acc29f534ffa168b" Jul 09 04:11:29.246744 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:29.246726 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-k8dnq_openshift-console-operator(633a3ae5-3d78-4217-901a-6d0d06442cf4)\"" pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" podUID="633a3ae5-3d78-4217-901a-6d0d06442cf4" Jul 09 04:11:31.772653 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:31.772613 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" podUID="8b49d1ca-6e2d-4701-b291-a1a9c9810431" Jul 09 04:11:31.772653 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:31.772615 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-6prtw" podUID="675fded2-27b6-4bda-83ad-84f46e175548" Jul 09 04:11:31.772653 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:31.772621 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" podUID="9d2284ee-ac3c-4e3c-9b08-67abc9f14e75" Jul 09 04:11:32.090613 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:32.090582 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:11:32.090782 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:32.090623 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6prtw" Jul 09 04:11:32.090782 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:32.090584 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" Jul 09 04:11:32.729138 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:32.729102 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/85856d16-c1e7-4990-9982-6e01d8ebc7f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9hgjm\" (UID: \"85856d16-c1e7-4990-9982-6e01d8ebc7f6\") " pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:32.729368 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:32.729155 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/85856d16-c1e7-4990-9982-6e01d8ebc7f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9hgjm\" (UID: \"85856d16-c1e7-4990-9982-6e01d8ebc7f6\") " pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:32.729368 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:32.729221 2534 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 04:11:32.729368 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:32.729266 2534 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 04:11:32.729368 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:32.729299 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/85856d16-c1e7-4990-9982-6e01d8ebc7f6-kube-rbac-proxy-cm podName:85856d16-c1e7-4990-9982-6e01d8ebc7f6 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:40.729266241 +0000 UTC m=+290.952884710 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/85856d16-c1e7-4990-9982-6e01d8ebc7f6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9hgjm" (UID: "85856d16-c1e7-4990-9982-6e01d8ebc7f6") : configmap "kube-rbac-proxy" not found Jul 09 04:11:32.729368 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:32.729355 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85856d16-c1e7-4990-9982-6e01d8ebc7f6-insights-runtime-extractor-tls podName:85856d16-c1e7-4990-9982-6e01d8ebc7f6 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:40.729339034 +0000 UTC m=+290.952957511 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/85856d16-c1e7-4990-9982-6e01d8ebc7f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9hgjm" (UID: "85856d16-c1e7-4990-9982-6e01d8ebc7f6") : secret "insights-runtime-extractor-tls" not found Jul 09 04:11:34.847656 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:34.847626 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ad788a33-5689-40d9-89d1-5200f27e9549-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9w76h\" (UID: \"ad788a33-5689-40d9-89d1-5200f27e9549\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9w76h" Jul 09 04:11:34.850155 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:34.850137 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ad788a33-5689-40d9-89d1-5200f27e9549-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9w76h\" (UID: \"ad788a33-5689-40d9-89d1-5200f27e9549\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9w76h" Jul 09 04:11:34.933150 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:34.933127 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9w76h" Jul 09 04:11:35.058479 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:35.058452 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9w76h"] Jul 09 04:11:35.100432 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:35.100410 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9w76h" event={"ID":"ad788a33-5689-40d9-89d1-5200f27e9549","Type":"ContainerStarted","Data":"9867ca8b614586d29226d28187ccfdb6eb84b90186be11154107fe9c60c3f233"} Jul 09 04:11:35.654324 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:35.654273 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:11:35.654483 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:35.654342 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls\") pod \"dns-default-6prtw\" (UID: \"675fded2-27b6-4bda-83ad-84f46e175548\") " pod="openshift-dns/dns-default-6prtw" Jul 09 04:11:35.654483 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:35.654393 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-z5c2w\" (UID: \"9d2284ee-ac3c-4e3c-9b08-67abc9f14e75\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" Jul 09 04:11:35.654552 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:35.654534 2534 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 04:11:35.654616 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:35.654606 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert podName:9d2284ee-ac3c-4e3c-9b08-67abc9f14e75 nodeName:}" failed. No retries permitted until 2026-07-09 04:13:37.654587591 +0000 UTC m=+407.878206061 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-z5c2w" (UID: "9d2284ee-ac3c-4e3c-9b08-67abc9f14e75") : secret "networking-console-plugin-cert" not found Jul 09 04:11:35.656848 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:35.656819 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/675fded2-27b6-4bda-83ad-84f46e175548-metrics-tls\") pod \"dns-default-6prtw\" (UID: \"675fded2-27b6-4bda-83ad-84f46e175548\") " pod="openshift-dns/dns-default-6prtw" Jul 09 04:11:35.656987 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:35.656872 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls\") pod \"image-registry-6cd88777bb-gm7kj\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:11:35.694847 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:35.694822 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mbnmf\"" Jul 09 04:11:35.694847 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:35.694822 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fd6z5\"" Jul 09 04:11:35.702222 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:35.702203 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6prtw" Jul 09 04:11:35.702335 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:35.702222 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:11:35.841714 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:35.841685 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cd88777bb-gm7kj"] Jul 09 04:11:35.845863 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:11:35.845840 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b49d1ca_6e2d_4701_b291_a1a9c9810431.slice/crio-e077245447f41eedf3ce98dd2b681047d8c59a80f2090d34bea65baf5b727697 WatchSource:0}: Error finding container e077245447f41eedf3ce98dd2b681047d8c59a80f2090d34bea65baf5b727697: Status 404 returned error can't find the container with id e077245447f41eedf3ce98dd2b681047d8c59a80f2090d34bea65baf5b727697 Jul 09 04:11:35.856747 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:35.856722 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6prtw"] Jul 09 04:11:35.859403 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:11:35.859380 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod675fded2_27b6_4bda_83ad_84f46e175548.slice/crio-80c46ba0e979825a047521b41c2558b600fb49b5bfbdc912647052fa1e969818 WatchSource:0}: Error finding container 80c46ba0e979825a047521b41c2558b600fb49b5bfbdc912647052fa1e969818: Status 404 returned error can't find the container with id 80c46ba0e979825a047521b41c2558b600fb49b5bfbdc912647052fa1e969818 Jul 09 04:11:36.104890 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:36.104845 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6prtw" event={"ID":"675fded2-27b6-4bda-83ad-84f46e175548","Type":"ContainerStarted","Data":"80c46ba0e979825a047521b41c2558b600fb49b5bfbdc912647052fa1e969818"} Jul 09 04:11:36.106454 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:36.106423 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" event={"ID":"8b49d1ca-6e2d-4701-b291-a1a9c9810431","Type":"ContainerStarted","Data":"6b750120dc81fc9d69bbecad754bd42fd972968a31d2de2670764c75de869f62"} Jul 09 04:11:36.106583 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:36.106478 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" event={"ID":"8b49d1ca-6e2d-4701-b291-a1a9c9810431","Type":"ContainerStarted","Data":"e077245447f41eedf3ce98dd2b681047d8c59a80f2090d34bea65baf5b727697"} Jul 09 04:11:36.106583 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:36.106543 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:11:36.124715 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:36.124656 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" podStartSLOduration=285.12463301 podStartE2EDuration="4m45.12463301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:06:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:11:36.122836755 +0000 UTC m=+286.346455247" watchObservedRunningTime="2026-07-09 04:11:36.12463301 +0000 UTC m=+286.348251499" Jul 09 04:11:38.118798 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:38.118758 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6prtw" event={"ID":"675fded2-27b6-4bda-83ad-84f46e175548","Type":"ContainerStarted","Data":"be9ec604c2ff75c68aa2bf52da73230834161cd34726926151253466c9b450c3"} Jul 09 04:11:38.119235 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:38.118815 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6prtw" event={"ID":"675fded2-27b6-4bda-83ad-84f46e175548","Type":"ContainerStarted","Data":"7fce78c75d9c36ef605f9ac2e3b372f2177b775c3ebd4af9c0272106136070e6"} Jul 09 04:11:38.119235 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:38.118845 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6prtw" Jul 09 04:11:38.120396 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:38.120371 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9w76h" event={"ID":"ad788a33-5689-40d9-89d1-5200f27e9549","Type":"ContainerStarted","Data":"3f5c4fc6ada5acba2a19f0790490e32ef72bee72c3af4a03b002779fc96d5e2d"} Jul 09 04:11:38.120499 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:38.120414 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9w76h" event={"ID":"ad788a33-5689-40d9-89d1-5200f27e9549","Type":"ContainerStarted","Data":"554e09d41fe02e9254816525546121dd08b77aecb131d46eee1fa2258a7a8046"} Jul 09 04:11:38.135030 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:38.134941 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6prtw" podStartSLOduration=251.109746442 podStartE2EDuration="4m13.134929465s" totalImagesPullingTime="2.025183023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:07:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:11:38.133821298 +0000 UTC m=+288.357439786" watchObservedRunningTime="2026-07-09 04:11:38.134929465 +0000 UTC m=+288.358547953" Jul 09 04:11:38.148244 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:38.148204 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9w76h" podStartSLOduration=17.357635488 podStartE2EDuration="20.14819388s" totalImagesPullingTime="2.790558392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:11:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:11:38.147489098 +0000 UTC m=+288.371107586" watchObservedRunningTime="2026-07-09 04:11:38.14819388 +0000 UTC m=+288.371812367" Jul 09 04:11:40.794497 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:40.794461 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/85856d16-c1e7-4990-9982-6e01d8ebc7f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9hgjm\" (UID: \"85856d16-c1e7-4990-9982-6e01d8ebc7f6\") " pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:40.794948 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:40.794528 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/85856d16-c1e7-4990-9982-6e01d8ebc7f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9hgjm\" (UID: \"85856d16-c1e7-4990-9982-6e01d8ebc7f6\") " pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:40.795151 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:40.795129 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/85856d16-c1e7-4990-9982-6e01d8ebc7f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9hgjm\" (UID: \"85856d16-c1e7-4990-9982-6e01d8ebc7f6\") " pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:40.796898 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:40.796875 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/85856d16-c1e7-4990-9982-6e01d8ebc7f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9hgjm\" (UID: \"85856d16-c1e7-4990-9982-6e01d8ebc7f6\") " pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:40.799652 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:40.799631 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9hgjm" Jul 09 04:11:40.939809 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:40.939646 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9hgjm"] Jul 09 04:11:40.942299 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:11:40.942261 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85856d16_c1e7_4990_9982_6e01d8ebc7f6.slice/crio-37f60bd3208dd2c97a0704ecd65dd34d484c44e41e57409e53c7cc787dea8d7d WatchSource:0}: Error finding container 37f60bd3208dd2c97a0704ecd65dd34d484c44e41e57409e53c7cc787dea8d7d: Status 404 returned error can't find the container with id 37f60bd3208dd2c97a0704ecd65dd34d484c44e41e57409e53c7cc787dea8d7d Jul 09 04:11:41.132514 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:41.132428 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9hgjm" event={"ID":"85856d16-c1e7-4990-9982-6e01d8ebc7f6","Type":"ContainerStarted","Data":"37f60bd3208dd2c97a0704ecd65dd34d484c44e41e57409e53c7cc787dea8d7d"} Jul 09 04:11:41.245199 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:41.245174 2534 scope.go:117] "RemoveContainer" containerID="b7ff8a77bac5aa0a21e538caf32b72ad0d9e20ffa2150627acc29f534ffa168b" Jul 09 04:11:42.139216 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:42.139188 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8dnq_633a3ae5-3d78-4217-901a-6d0d06442cf4/console-operator/1.log" Jul 09 04:11:42.139667 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:42.139259 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" event={"ID":"633a3ae5-3d78-4217-901a-6d0d06442cf4","Type":"ContainerStarted","Data":"63daf2a5134350e99d653b33327634e33bd355cc50444e8f1e4977378c11b4d1"} Jul 09 04:11:42.139667 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:42.139590 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" Jul 09 04:11:42.155648 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:42.155603 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" podStartSLOduration=20.056735745 podStartE2EDuration="24.155584239s" totalImagesPullingTime="4.098848494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:11:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:11:42.154555542 +0000 UTC m=+292.378174031" watchObservedRunningTime="2026-07-09 04:11:42.155584239 +0000 UTC m=+292.379202728" Jul 09 04:11:42.644190 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:42.644160 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-k8dnq" Jul 09 04:11:44.245641 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:44.245603 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-b4gcd" podUID="9fefcefc-4128-4422-98de-b17e5949cfc7" Jul 09 04:11:45.157393 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:45.157301 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9hgjm" event={"ID":"85856d16-c1e7-4990-9982-6e01d8ebc7f6","Type":"ContainerStarted","Data":"b1cee6ece86908620dcece3a178925199d38df7e47d3a7032090987b446c471c"} Jul 09 04:11:46.163819 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:46.163778 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9hgjm" event={"ID":"85856d16-c1e7-4990-9982-6e01d8ebc7f6","Type":"ContainerStarted","Data":"36acc9c678f001f26a6e2117c8483f50d303f0e7192e998a6ef67ddc6ca5c502"} Jul 09 04:11:48.126440 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:48.126414 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6prtw" Jul 09 04:11:48.172602 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:48.172566 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9hgjm" event={"ID":"85856d16-c1e7-4990-9982-6e01d8ebc7f6","Type":"ContainerStarted","Data":"e9bbbb157ad19b77c8fb4595db2f24de3ee31bee4128d058f642efef02b2286e"} Jul 09 04:11:48.189729 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:48.189674 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9hgjm" podStartSLOduration=17.244565436 podStartE2EDuration="24.189657744s" totalImagesPullingTime="6.945092308s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:11:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:11:48.188665653 +0000 UTC m=+298.412284142" watchObservedRunningTime="2026-07-09 04:11:48.189657744 +0000 UTC m=+298.413276251" Jul 09 04:11:48.699439 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:48.699406 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-6q5mp"] Jul 09 04:11:48.701605 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:48.701584 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-6q5mp" Jul 09 04:11:48.703912 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:48.703888 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 09 04:11:48.704293 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:48.704247 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-49mtp\"" Jul 09 04:11:48.704400 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:48.704307 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 09 04:11:48.713260 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:48.713228 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-6q5mp"] Jul 09 04:11:48.763572 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:48.763543 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-967nc\" (UniqueName: \"kubernetes.io/projected/e539f6ff-e687-4517-b313-96a208ec1d48-kube-api-access-967nc\") pod \"downloads-685bfb5f64-6q5mp\" (UID: \"e539f6ff-e687-4517-b313-96a208ec1d48\") " pod="openshift-console/downloads-685bfb5f64-6q5mp" Jul 09 04:11:48.864526 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:48.864492 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-967nc\" (UniqueName: \"kubernetes.io/projected/e539f6ff-e687-4517-b313-96a208ec1d48-kube-api-access-967nc\") pod \"downloads-685bfb5f64-6q5mp\" (UID: \"e539f6ff-e687-4517-b313-96a208ec1d48\") " pod="openshift-console/downloads-685bfb5f64-6q5mp" Jul 09 04:11:48.874723 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:48.874697 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-967nc\" (UniqueName: \"kubernetes.io/projected/e539f6ff-e687-4517-b313-96a208ec1d48-kube-api-access-967nc\") pod \"downloads-685bfb5f64-6q5mp\" (UID: \"e539f6ff-e687-4517-b313-96a208ec1d48\") " pod="openshift-console/downloads-685bfb5f64-6q5mp" Jul 09 04:11:49.010892 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:49.010803 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-6q5mp" Jul 09 04:11:49.141627 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:49.141596 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-6q5mp"] Jul 09 04:11:49.144410 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:11:49.144385 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode539f6ff_e687_4517_b313_96a208ec1d48.slice/crio-413cc3c24019dcdbb664ff0d9755bcd4d170c906a05e8da143315b798ce653b7 WatchSource:0}: Error finding container 413cc3c24019dcdbb664ff0d9755bcd4d170c906a05e8da143315b798ce653b7: Status 404 returned error can't find the container with id 413cc3c24019dcdbb664ff0d9755bcd4d170c906a05e8da143315b798ce653b7 Jul 09 04:11:49.176939 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:49.176904 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-6q5mp" event={"ID":"e539f6ff-e687-4517-b313-96a208ec1d48","Type":"ContainerStarted","Data":"413cc3c24019dcdbb664ff0d9755bcd4d170c906a05e8da143315b798ce653b7"} Jul 09 04:11:50.151598 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:50.151568 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8dnq_633a3ae5-3d78-4217-901a-6d0d06442cf4/console-operator/1.log" Jul 09 04:11:50.152058 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:50.151635 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8dnq_633a3ae5-3d78-4217-901a-6d0d06442cf4/console-operator/1.log" Jul 09 04:11:50.171147 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:50.171124 2534 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 04:11:55.708825 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:55.708778 2534 patch_prober.go:28] interesting pod/image-registry-6cd88777bb-gm7kj container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 09 04:11:55.710727 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:55.708848 2534 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" podUID="8b49d1ca-6e2d-4701-b291-a1a9c9810431" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 04:11:56.925661 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:56.925628 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-nzn6j"] Jul 09 04:11:56.927989 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:56.927971 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-nzn6j" Jul 09 04:11:56.930642 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:56.930452 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 09 04:11:56.930642 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:56.930468 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 04:11:56.930642 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:56.930517 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 09 04:11:56.930642 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:56.930549 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 04:11:56.931624 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:56.931586 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-nrw77\"" Jul 09 04:11:56.931737 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:56.931727 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 04:11:56.936593 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:56.936572 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-nzn6j"] Jul 09 04:11:57.036875 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:57.036847 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81de11f1-ca34-4c0a-be5d-5ff455159cbb-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-nzn6j\" (UID: \"81de11f1-ca34-4c0a-be5d-5ff455159cbb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-nzn6j" Jul 09 04:11:57.037129 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:57.036919 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/81de11f1-ca34-4c0a-be5d-5ff455159cbb-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-nzn6j\" (UID: \"81de11f1-ca34-4c0a-be5d-5ff455159cbb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-nzn6j" Jul 09 04:11:57.037129 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:57.036947 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/81de11f1-ca34-4c0a-be5d-5ff455159cbb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-nzn6j\" (UID: \"81de11f1-ca34-4c0a-be5d-5ff455159cbb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-nzn6j" Jul 09 04:11:57.037129 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:57.036984 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k64wd\" (UniqueName: \"kubernetes.io/projected/81de11f1-ca34-4c0a-be5d-5ff455159cbb-kube-api-access-k64wd\") pod \"prometheus-operator-567f7b8b7c-nzn6j\" (UID: \"81de11f1-ca34-4c0a-be5d-5ff455159cbb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-nzn6j" Jul 09 04:11:57.116154 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:57.116128 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:11:57.137359 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:57.137331 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81de11f1-ca34-4c0a-be5d-5ff455159cbb-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-nzn6j\" (UID: \"81de11f1-ca34-4c0a-be5d-5ff455159cbb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-nzn6j" Jul 09 04:11:57.137536 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:57.137393 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/81de11f1-ca34-4c0a-be5d-5ff455159cbb-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-nzn6j\" (UID: \"81de11f1-ca34-4c0a-be5d-5ff455159cbb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-nzn6j" Jul 09 04:11:57.137536 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:57.137431 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/81de11f1-ca34-4c0a-be5d-5ff455159cbb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-nzn6j\" (UID: \"81de11f1-ca34-4c0a-be5d-5ff455159cbb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-nzn6j" Jul 09 04:11:57.137536 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:57.137472 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k64wd\" (UniqueName: \"kubernetes.io/projected/81de11f1-ca34-4c0a-be5d-5ff455159cbb-kube-api-access-k64wd\") pod \"prometheus-operator-567f7b8b7c-nzn6j\" (UID: \"81de11f1-ca34-4c0a-be5d-5ff455159cbb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-nzn6j" Jul 09 04:11:57.137701 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:57.137577 2534 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 09 04:11:57.137701 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:11:57.137647 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81de11f1-ca34-4c0a-be5d-5ff455159cbb-prometheus-operator-tls podName:81de11f1-ca34-4c0a-be5d-5ff455159cbb nodeName:}" failed. No retries permitted until 2026-07-09 04:11:57.637627835 +0000 UTC m=+307.861246305 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/81de11f1-ca34-4c0a-be5d-5ff455159cbb-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-nzn6j" (UID: "81de11f1-ca34-4c0a-be5d-5ff455159cbb") : secret "prometheus-operator-tls" not found Jul 09 04:11:57.138117 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:57.138094 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81de11f1-ca34-4c0a-be5d-5ff455159cbb-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-nzn6j\" (UID: \"81de11f1-ca34-4c0a-be5d-5ff455159cbb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-nzn6j" Jul 09 04:11:57.140071 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:57.140050 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/81de11f1-ca34-4c0a-be5d-5ff455159cbb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-nzn6j\" (UID: \"81de11f1-ca34-4c0a-be5d-5ff455159cbb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-nzn6j" Jul 09 04:11:57.148381 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:57.148357 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k64wd\" (UniqueName: \"kubernetes.io/projected/81de11f1-ca34-4c0a-be5d-5ff455159cbb-kube-api-access-k64wd\") pod \"prometheus-operator-567f7b8b7c-nzn6j\" (UID: \"81de11f1-ca34-4c0a-be5d-5ff455159cbb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-nzn6j" Jul 09 04:11:57.244361 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:57.244274 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:11:57.641140 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:57.641109 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/81de11f1-ca34-4c0a-be5d-5ff455159cbb-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-nzn6j\" (UID: \"81de11f1-ca34-4c0a-be5d-5ff455159cbb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-nzn6j" Jul 09 04:11:57.643851 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:57.643817 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/81de11f1-ca34-4c0a-be5d-5ff455159cbb-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-nzn6j\" (UID: \"81de11f1-ca34-4c0a-be5d-5ff455159cbb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-nzn6j" Jul 09 04:11:57.838604 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:57.838571 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-nzn6j" Jul 09 04:11:57.981113 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:57.981089 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-nzn6j"] Jul 09 04:11:57.983753 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:11:57.983728 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81de11f1_ca34_4c0a_be5d_5ff455159cbb.slice/crio-cd9057870fa8cf4065a3d7499e0a5d853c5bbe31f82b98767d3b17b04ea44716 WatchSource:0}: Error finding container cd9057870fa8cf4065a3d7499e0a5d853c5bbe31f82b98767d3b17b04ea44716: Status 404 returned error can't find the container with id cd9057870fa8cf4065a3d7499e0a5d853c5bbe31f82b98767d3b17b04ea44716 Jul 09 04:11:57.986151 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:57.986134 2534 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 04:11:58.213410 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:11:58.213326 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-nzn6j" event={"ID":"81de11f1-ca34-4c0a-be5d-5ff455159cbb","Type":"ContainerStarted","Data":"cd9057870fa8cf4065a3d7499e0a5d853c5bbe31f82b98767d3b17b04ea44716"} Jul 09 04:12:00.222901 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:00.222861 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-nzn6j" event={"ID":"81de11f1-ca34-4c0a-be5d-5ff455159cbb","Type":"ContainerStarted","Data":"a9799615b2b80a0c040c65768a74b6b2a119cd88559d17f5e8b61ee591a8fde5"} Jul 09 04:12:00.223313 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:00.222941 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-nzn6j" event={"ID":"81de11f1-ca34-4c0a-be5d-5ff455159cbb","Type":"ContainerStarted","Data":"9e64e55961db4d6ffaab12ba4d5793d6897668b3050eb61118de3cbeb1efc23a"} Jul 09 04:12:00.239711 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:00.239661 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-nzn6j" podStartSLOduration=2.749090682 podStartE2EDuration="4.239645254s" totalImagesPullingTime="1.490554572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:11:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:12:00.238480678 +0000 UTC m=+310.462099167" watchObservedRunningTime="2026-07-09 04:12:00.239645254 +0000 UTC m=+310.463263754" Jul 09 04:12:02.284437 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.284405 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-xrj2l"] Jul 09 04:12:02.287396 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.287360 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xrj2l" Jul 09 04:12:02.290309 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.290200 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 09 04:12:02.290309 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.290220 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 09 04:12:02.290309 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.290224 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-wh4k5\"" Jul 09 04:12:02.299434 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.299413 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vljg2"] Jul 09 04:12:02.302013 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.301983 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-xrj2l"] Jul 09 04:12:02.302125 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.302108 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.302482 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.302457 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4"] Jul 09 04:12:02.304399 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.304379 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 04:12:02.304560 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.304546 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 04:12:02.304645 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.304594 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 04:12:02.304828 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.304811 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rzmb5\"" Jul 09 04:12:02.305825 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.305807 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4" Jul 09 04:12:02.308166 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.308148 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 09 04:12:02.308166 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.308163 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-rhcxn\"" Jul 09 04:12:02.308362 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.308179 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 09 04:12:02.308362 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.308342 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 09 04:12:02.318363 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.318338 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4"] Jul 09 04:12:02.386829 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.386792 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/742d0176-a6e5-4a5a-932b-460367548241-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-4cnc4\" (UID: \"742d0176-a6e5-4a5a-932b-460367548241\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4" Jul 09 04:12:02.387011 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.386849 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-node-exporter-wtmp\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.387011 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.386879 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/742d0176-a6e5-4a5a-932b-460367548241-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-4cnc4\" (UID: \"742d0176-a6e5-4a5a-932b-460367548241\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4" Jul 09 04:12:02.387011 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.386939 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzmcr\" (UniqueName: \"kubernetes.io/projected/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-kube-api-access-fzmcr\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.387011 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.386985 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/742d0176-a6e5-4a5a-932b-460367548241-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-4cnc4\" (UID: \"742d0176-a6e5-4a5a-932b-460367548241\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4" Jul 09 04:12:02.387183 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.387019 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.387183 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.387054 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-metrics-client-ca\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.387183 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.387120 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/52f790aa-4d62-4d1f-afee-2fb4aa845265-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-xrj2l\" (UID: \"52f790aa-4d62-4d1f-afee-2fb4aa845265\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xrj2l" Jul 09 04:12:02.387183 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.387144 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52f790aa-4d62-4d1f-afee-2fb4aa845265-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-xrj2l\" (UID: \"52f790aa-4d62-4d1f-afee-2fb4aa845265\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xrj2l" Jul 09 04:12:02.387183 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.387162 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/742d0176-a6e5-4a5a-932b-460367548241-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-4cnc4\" (UID: \"742d0176-a6e5-4a5a-932b-460367548241\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4" Jul 09 04:12:02.387460 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.387235 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-node-exporter-accelerators-collector-config\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.387460 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.387273 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-227sd\" (UniqueName: \"kubernetes.io/projected/52f790aa-4d62-4d1f-afee-2fb4aa845265-kube-api-access-227sd\") pod \"openshift-state-metrics-54cf96dfcf-xrj2l\" (UID: \"52f790aa-4d62-4d1f-afee-2fb4aa845265\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xrj2l" Jul 09 04:12:02.387460 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.387321 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-root\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.387460 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.387342 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-node-exporter-textfile\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.387460 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.387363 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/742d0176-a6e5-4a5a-932b-460367548241-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-4cnc4\" (UID: \"742d0176-a6e5-4a5a-932b-460367548241\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4" Jul 09 04:12:02.387460 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.387388 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-node-exporter-tls\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.387460 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.387416 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52f790aa-4d62-4d1f-afee-2fb4aa845265-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-xrj2l\" (UID: \"52f790aa-4d62-4d1f-afee-2fb4aa845265\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xrj2l" Jul 09 04:12:02.387460 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.387432 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgk29\" (UniqueName: \"kubernetes.io/projected/742d0176-a6e5-4a5a-932b-460367548241-kube-api-access-fgk29\") pod \"kube-state-metrics-7c9dbb8b7d-4cnc4\" (UID: \"742d0176-a6e5-4a5a-932b-460367548241\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4" Jul 09 04:12:02.387460 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.387449 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-sys\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.488500 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.488467 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-node-exporter-wtmp\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.488500 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.488505 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/742d0176-a6e5-4a5a-932b-460367548241-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-4cnc4\" (UID: \"742d0176-a6e5-4a5a-932b-460367548241\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4" Jul 09 04:12:02.488749 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.488540 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzmcr\" (UniqueName: \"kubernetes.io/projected/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-kube-api-access-fzmcr\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.488749 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.488572 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/742d0176-a6e5-4a5a-932b-460367548241-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-4cnc4\" (UID: \"742d0176-a6e5-4a5a-932b-460367548241\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4" Jul 09 04:12:02.488749 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.488609 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.488749 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.488643 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-metrics-client-ca\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.488749 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.488677 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/52f790aa-4d62-4d1f-afee-2fb4aa845265-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-xrj2l\" (UID: \"52f790aa-4d62-4d1f-afee-2fb4aa845265\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xrj2l" Jul 09 04:12:02.488749 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.488678 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-node-exporter-wtmp\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.488749 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.488732 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52f790aa-4d62-4d1f-afee-2fb4aa845265-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-xrj2l\" (UID: \"52f790aa-4d62-4d1f-afee-2fb4aa845265\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xrj2l" Jul 09 04:12:02.489093 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.488766 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/742d0176-a6e5-4a5a-932b-460367548241-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-4cnc4\" (UID: \"742d0176-a6e5-4a5a-932b-460367548241\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4" Jul 09 04:12:02.489093 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.488810 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-node-exporter-accelerators-collector-config\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.489093 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.488838 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-227sd\" (UniqueName: \"kubernetes.io/projected/52f790aa-4d62-4d1f-afee-2fb4aa845265-kube-api-access-227sd\") pod \"openshift-state-metrics-54cf96dfcf-xrj2l\" (UID: \"52f790aa-4d62-4d1f-afee-2fb4aa845265\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xrj2l" Jul 09 04:12:02.489093 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.488867 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-root\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.489093 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.488892 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-node-exporter-textfile\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.489093 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.488922 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/742d0176-a6e5-4a5a-932b-460367548241-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-4cnc4\" (UID: \"742d0176-a6e5-4a5a-932b-460367548241\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4" Jul 09 04:12:02.489093 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.488947 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-node-exporter-tls\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.489093 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.488973 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52f790aa-4d62-4d1f-afee-2fb4aa845265-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-xrj2l\" (UID: \"52f790aa-4d62-4d1f-afee-2fb4aa845265\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xrj2l" Jul 09 04:12:02.489093 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.489002 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgk29\" (UniqueName: \"kubernetes.io/projected/742d0176-a6e5-4a5a-932b-460367548241-kube-api-access-fgk29\") pod \"kube-state-metrics-7c9dbb8b7d-4cnc4\" (UID: \"742d0176-a6e5-4a5a-932b-460367548241\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4" Jul 09 04:12:02.489093 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.489025 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-sys\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.489093 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.489041 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/742d0176-a6e5-4a5a-932b-460367548241-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-4cnc4\" (UID: \"742d0176-a6e5-4a5a-932b-460367548241\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4" Jul 09 04:12:02.489093 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.489076 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/742d0176-a6e5-4a5a-932b-460367548241-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-4cnc4\" (UID: \"742d0176-a6e5-4a5a-932b-460367548241\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4" Jul 09 04:12:02.489701 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.489163 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-root\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.489701 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.489423 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-sys\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.489701 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.489545 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-node-exporter-accelerators-collector-config\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.489701 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.489565 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-metrics-client-ca\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.490057 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.489726 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/742d0176-a6e5-4a5a-932b-460367548241-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-4cnc4\" (UID: \"742d0176-a6e5-4a5a-932b-460367548241\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4" Jul 09 04:12:02.490057 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.489978 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-node-exporter-textfile\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.490520 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.490262 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/742d0176-a6e5-4a5a-932b-460367548241-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-4cnc4\" (UID: \"742d0176-a6e5-4a5a-932b-460367548241\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4" Jul 09 04:12:02.490520 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.490424 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52f790aa-4d62-4d1f-afee-2fb4aa845265-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-xrj2l\" (UID: \"52f790aa-4d62-4d1f-afee-2fb4aa845265\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xrj2l" Jul 09 04:12:02.492509 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.492463 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.492744 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.492609 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52f790aa-4d62-4d1f-afee-2fb4aa845265-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-xrj2l\" (UID: \"52f790aa-4d62-4d1f-afee-2fb4aa845265\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xrj2l" Jul 09 04:12:02.493000 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.492982 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-node-exporter-tls\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.493083 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.493010 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/742d0176-a6e5-4a5a-932b-460367548241-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-4cnc4\" (UID: \"742d0176-a6e5-4a5a-932b-460367548241\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4" Jul 09 04:12:02.493471 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.493451 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/742d0176-a6e5-4a5a-932b-460367548241-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-4cnc4\" (UID: \"742d0176-a6e5-4a5a-932b-460367548241\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4" Jul 09 04:12:02.494953 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.494928 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/52f790aa-4d62-4d1f-afee-2fb4aa845265-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-xrj2l\" (UID: \"52f790aa-4d62-4d1f-afee-2fb4aa845265\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xrj2l" Jul 09 04:12:02.512934 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.512906 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-227sd\" (UniqueName: \"kubernetes.io/projected/52f790aa-4d62-4d1f-afee-2fb4aa845265-kube-api-access-227sd\") pod \"openshift-state-metrics-54cf96dfcf-xrj2l\" (UID: \"52f790aa-4d62-4d1f-afee-2fb4aa845265\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xrj2l" Jul 09 04:12:02.513070 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.512963 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzmcr\" (UniqueName: \"kubernetes.io/projected/0a316c28-7d81-4e36-bff6-7ea5ef5ab248-kube-api-access-fzmcr\") pod \"node-exporter-vljg2\" (UID: \"0a316c28-7d81-4e36-bff6-7ea5ef5ab248\") " pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.513137 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.513067 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgk29\" (UniqueName: \"kubernetes.io/projected/742d0176-a6e5-4a5a-932b-460367548241-kube-api-access-fgk29\") pod \"kube-state-metrics-7c9dbb8b7d-4cnc4\" (UID: \"742d0176-a6e5-4a5a-932b-460367548241\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4" Jul 09 04:12:02.599448 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.599403 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xrj2l" Jul 09 04:12:02.619462 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.619431 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vljg2" Jul 09 04:12:02.625411 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:02.625390 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4" Jul 09 04:12:05.308322 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.308275 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr"] Jul 09 04:12:05.311478 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.311446 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:05.313907 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.313861 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 09 04:12:05.314055 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.313925 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 09 04:12:05.314055 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.313864 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 09 04:12:05.314055 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.313864 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-8vsh9\"" Jul 09 04:12:05.314055 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.313869 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 09 04:12:05.314055 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.313957 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-dh3i5nohn4d1b\"" Jul 09 04:12:05.314368 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.314334 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 09 04:12:05.322483 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.322461 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr"] Jul 09 04:12:05.415727 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.415675 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3fa09ac0-300c-4d25-baa5-e07c192168e8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8fd4cf69b-c4gtr\" (UID: \"3fa09ac0-300c-4d25-baa5-e07c192168e8\") " pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:05.415727 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.415721 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3fa09ac0-300c-4d25-baa5-e07c192168e8-secret-grpc-tls\") pod \"thanos-querier-8fd4cf69b-c4gtr\" (UID: \"3fa09ac0-300c-4d25-baa5-e07c192168e8\") " pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:05.415947 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.415788 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3fa09ac0-300c-4d25-baa5-e07c192168e8-secret-thanos-querier-tls\") pod \"thanos-querier-8fd4cf69b-c4gtr\" (UID: \"3fa09ac0-300c-4d25-baa5-e07c192168e8\") " pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:05.415947 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.415818 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3fa09ac0-300c-4d25-baa5-e07c192168e8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8fd4cf69b-c4gtr\" (UID: \"3fa09ac0-300c-4d25-baa5-e07c192168e8\") " pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:05.415947 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.415846 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3fa09ac0-300c-4d25-baa5-e07c192168e8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8fd4cf69b-c4gtr\" (UID: \"3fa09ac0-300c-4d25-baa5-e07c192168e8\") " pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:05.415947 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.415877 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3fa09ac0-300c-4d25-baa5-e07c192168e8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8fd4cf69b-c4gtr\" (UID: \"3fa09ac0-300c-4d25-baa5-e07c192168e8\") " pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:05.415947 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.415933 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fa09ac0-300c-4d25-baa5-e07c192168e8-metrics-client-ca\") pod \"thanos-querier-8fd4cf69b-c4gtr\" (UID: \"3fa09ac0-300c-4d25-baa5-e07c192168e8\") " pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:05.416148 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.415989 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvg5x\" (UniqueName: \"kubernetes.io/projected/3fa09ac0-300c-4d25-baa5-e07c192168e8-kube-api-access-jvg5x\") pod \"thanos-querier-8fd4cf69b-c4gtr\" (UID: \"3fa09ac0-300c-4d25-baa5-e07c192168e8\") " pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:05.517052 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.517022 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3fa09ac0-300c-4d25-baa5-e07c192168e8-secret-grpc-tls\") pod \"thanos-querier-8fd4cf69b-c4gtr\" (UID: \"3fa09ac0-300c-4d25-baa5-e07c192168e8\") " pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:05.517245 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.517109 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3fa09ac0-300c-4d25-baa5-e07c192168e8-secret-thanos-querier-tls\") pod \"thanos-querier-8fd4cf69b-c4gtr\" (UID: \"3fa09ac0-300c-4d25-baa5-e07c192168e8\") " pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:05.517245 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.517139 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3fa09ac0-300c-4d25-baa5-e07c192168e8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8fd4cf69b-c4gtr\" (UID: \"3fa09ac0-300c-4d25-baa5-e07c192168e8\") " pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:05.517245 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.517164 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3fa09ac0-300c-4d25-baa5-e07c192168e8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8fd4cf69b-c4gtr\" (UID: \"3fa09ac0-300c-4d25-baa5-e07c192168e8\") " pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:05.517442 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.517337 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3fa09ac0-300c-4d25-baa5-e07c192168e8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8fd4cf69b-c4gtr\" (UID: \"3fa09ac0-300c-4d25-baa5-e07c192168e8\") " pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:05.517442 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.517434 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fa09ac0-300c-4d25-baa5-e07c192168e8-metrics-client-ca\") pod \"thanos-querier-8fd4cf69b-c4gtr\" (UID: \"3fa09ac0-300c-4d25-baa5-e07c192168e8\") " pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:05.517550 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.517491 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvg5x\" (UniqueName: \"kubernetes.io/projected/3fa09ac0-300c-4d25-baa5-e07c192168e8-kube-api-access-jvg5x\") pod \"thanos-querier-8fd4cf69b-c4gtr\" (UID: \"3fa09ac0-300c-4d25-baa5-e07c192168e8\") " pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:05.517550 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.517527 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3fa09ac0-300c-4d25-baa5-e07c192168e8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8fd4cf69b-c4gtr\" (UID: \"3fa09ac0-300c-4d25-baa5-e07c192168e8\") " pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:05.518393 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.518368 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fa09ac0-300c-4d25-baa5-e07c192168e8-metrics-client-ca\") pod \"thanos-querier-8fd4cf69b-c4gtr\" (UID: \"3fa09ac0-300c-4d25-baa5-e07c192168e8\") " pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:05.520268 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.520245 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3fa09ac0-300c-4d25-baa5-e07c192168e8-secret-grpc-tls\") pod \"thanos-querier-8fd4cf69b-c4gtr\" (UID: \"3fa09ac0-300c-4d25-baa5-e07c192168e8\") " pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:05.520630 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.520604 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3fa09ac0-300c-4d25-baa5-e07c192168e8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8fd4cf69b-c4gtr\" (UID: \"3fa09ac0-300c-4d25-baa5-e07c192168e8\") " pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:05.520781 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.520756 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3fa09ac0-300c-4d25-baa5-e07c192168e8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8fd4cf69b-c4gtr\" (UID: \"3fa09ac0-300c-4d25-baa5-e07c192168e8\") " pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:05.521045 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.521026 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3fa09ac0-300c-4d25-baa5-e07c192168e8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8fd4cf69b-c4gtr\" (UID: \"3fa09ac0-300c-4d25-baa5-e07c192168e8\") " pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:05.521238 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.521200 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3fa09ac0-300c-4d25-baa5-e07c192168e8-secret-thanos-querier-tls\") pod \"thanos-querier-8fd4cf69b-c4gtr\" (UID: \"3fa09ac0-300c-4d25-baa5-e07c192168e8\") " pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:05.521571 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.521540 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3fa09ac0-300c-4d25-baa5-e07c192168e8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8fd4cf69b-c4gtr\" (UID: \"3fa09ac0-300c-4d25-baa5-e07c192168e8\") " pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:05.526430 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.526411 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvg5x\" (UniqueName: \"kubernetes.io/projected/3fa09ac0-300c-4d25-baa5-e07c192168e8-kube-api-access-jvg5x\") pod \"thanos-querier-8fd4cf69b-c4gtr\" (UID: \"3fa09ac0-300c-4d25-baa5-e07c192168e8\") " pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:05.623936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:05.623841 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:08.499518 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.499073 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 04:12:08.502745 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.502720 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.505180 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.505157 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 09 04:12:08.505307 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.505163 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 09 04:12:08.505496 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.505476 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 09 04:12:08.505622 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.505481 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 09 04:12:08.506797 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.506302 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 09 04:12:08.506797 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.506341 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 09 04:12:08.506797 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.506357 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 09 04:12:08.506797 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.506277 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-n24b7\"" Jul 09 04:12:08.506797 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.506492 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 09 04:12:08.506797 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.506570 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 09 04:12:08.506797 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.506578 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 09 04:12:08.506797 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.506707 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 09 04:12:08.506797 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.506796 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6qhi3nl02ho85\"" Jul 09 04:12:08.507513 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.506860 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 09 04:12:08.507513 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.506888 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 09 04:12:08.511340 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.511263 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 09 04:12:08.513956 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.513791 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 09 04:12:08.523300 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.523253 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 04:12:08.618461 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:12:08.618426 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a316c28_7d81_4e36_bff6_7ea5ef5ab248.slice/crio-16b4d0c3705bb09ea34d65726550fecf07a50b37de6ea305d9f5d962af753d50 WatchSource:0}: Error finding container 16b4d0c3705bb09ea34d65726550fecf07a50b37de6ea305d9f5d962af753d50: Status 404 returned error can't find the container with id 16b4d0c3705bb09ea34d65726550fecf07a50b37de6ea305d9f5d962af753d50 Jul 09 04:12:08.653799 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.652799 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.653799 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.652851 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.653799 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.652893 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.653799 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.652931 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-web-config\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.653799 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.652961 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.653799 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.652985 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2433376d-b2a6-424d-a06a-bf39dbb88b72-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.653799 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.653011 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.653799 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.653045 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.653799 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.653075 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.653799 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.653106 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-config\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.653799 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.653129 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.653799 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.653172 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.653799 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.653200 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.653799 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.653240 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.653799 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.653261 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.653799 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.653317 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2433376d-b2a6-424d-a06a-bf39dbb88b72-config-out\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.654897 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.653341 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.654897 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.653371 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.654897 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.653402 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.654897 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.653445 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d8xw\" (UniqueName: \"kubernetes.io/projected/2433376d-b2a6-424d-a06a-bf39dbb88b72-kube-api-access-2d8xw\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.755260 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.754784 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.755260 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.754832 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-config\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.755260 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.755004 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.755260 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.755059 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.755260 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.755084 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.755260 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.755112 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.755260 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.755127 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.755260 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.755146 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2433376d-b2a6-424d-a06a-bf39dbb88b72-config-out\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.755260 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.755161 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.755260 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.755182 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.755260 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.755199 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.755260 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.755227 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d8xw\" (UniqueName: \"kubernetes.io/projected/2433376d-b2a6-424d-a06a-bf39dbb88b72-kube-api-access-2d8xw\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.755260 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.755261 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.755947 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.755275 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.755947 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.755914 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.757322 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.756800 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.757322 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.756967 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.759026 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.757891 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.759026 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.758733 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.759026 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.758975 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.759842 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.759326 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.759842 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.759385 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-web-config\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.759842 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.759426 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.759842 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.759460 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2433376d-b2a6-424d-a06a-bf39dbb88b72-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.759842 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.759486 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.759842 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.759528 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.759842 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.759633 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.761063 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.761035 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.761713 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.761656 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.762550 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.762453 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.762732 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.762674 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-config\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.763383 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.763209 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.763994 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.763952 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.765378 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.765337 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-web-config\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.766427 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.766385 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.766754 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.766716 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.767222 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.767158 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2433376d-b2a6-424d-a06a-bf39dbb88b72-config-out\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.767495 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.767459 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d8xw\" (UniqueName: \"kubernetes.io/projected/2433376d-b2a6-424d-a06a-bf39dbb88b72-kube-api-access-2d8xw\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.767689 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.767577 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2433376d-b2a6-424d-a06a-bf39dbb88b72-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.771777 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.771680 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.778237 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.774801 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4"] Jul 09 04:12:08.780902 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:12:08.780783 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod742d0176_a6e5_4a5a_932b_460367548241.slice/crio-c40ca2656c12c02f4e37bf9e29207d7ab3dec112aef6f827248c2ad22138c29e WatchSource:0}: Error finding container c40ca2656c12c02f4e37bf9e29207d7ab3dec112aef6f827248c2ad22138c29e: Status 404 returned error can't find the container with id c40ca2656c12c02f4e37bf9e29207d7ab3dec112aef6f827248c2ad22138c29e Jul 09 04:12:08.811956 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.811832 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-xrj2l"] Jul 09 04:12:08.815106 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:12:08.815073 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52f790aa_4d62_4d1f_afee_2fb4aa845265.slice/crio-85907bdcf0c5fe50100c7bfec5c24846bb93c2642075c5a318b7baa6d91df6e8 WatchSource:0}: Error finding container 85907bdcf0c5fe50100c7bfec5c24846bb93c2642075c5a318b7baa6d91df6e8: Status 404 returned error can't find the container with id 85907bdcf0c5fe50100c7bfec5c24846bb93c2642075c5a318b7baa6d91df6e8 Jul 09 04:12:08.820005 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.817453 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:08.854503 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:08.854405 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr"] Jul 09 04:12:08.859769 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:12:08.859637 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fa09ac0_300c_4d25_baa5_e07c192168e8.slice/crio-238cb550d548414b47afdd42258b32f0178423041bb1ecc080914eef3d75db95 WatchSource:0}: Error finding container 238cb550d548414b47afdd42258b32f0178423041bb1ecc080914eef3d75db95: Status 404 returned error can't find the container with id 238cb550d548414b47afdd42258b32f0178423041bb1ecc080914eef3d75db95 Jul 09 04:12:09.004368 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:09.003909 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 04:12:09.012694 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:12:09.012662 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2433376d_b2a6_424d_a06a_bf39dbb88b72.slice/crio-a88bc29d177d6e2dabccef7c2e673dd176876d945cb731c2d13bd7159cc7ea34 WatchSource:0}: Error finding container a88bc29d177d6e2dabccef7c2e673dd176876d945cb731c2d13bd7159cc7ea34: Status 404 returned error can't find the container with id a88bc29d177d6e2dabccef7c2e673dd176876d945cb731c2d13bd7159cc7ea34 Jul 09 04:12:09.267091 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:09.266988 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xrj2l" event={"ID":"52f790aa-4d62-4d1f-afee-2fb4aa845265","Type":"ContainerStarted","Data":"deac42917edd85a519d49061cfc10c8341fd41aeff33c0132318a494af6d925d"} Jul 09 04:12:09.267091 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:09.267073 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xrj2l" event={"ID":"52f790aa-4d62-4d1f-afee-2fb4aa845265","Type":"ContainerStarted","Data":"fc46d8f4cb165ee13fbd0238ebb0521f9ad587dda98b54c3151f1e9599cc7255"} Jul 09 04:12:09.267091 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:09.267093 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xrj2l" event={"ID":"52f790aa-4d62-4d1f-afee-2fb4aa845265","Type":"ContainerStarted","Data":"85907bdcf0c5fe50100c7bfec5c24846bb93c2642075c5a318b7baa6d91df6e8"} Jul 09 04:12:09.268535 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:09.268506 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2433376d-b2a6-424d-a06a-bf39dbb88b72","Type":"ContainerStarted","Data":"a88bc29d177d6e2dabccef7c2e673dd176876d945cb731c2d13bd7159cc7ea34"} Jul 09 04:12:09.269945 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:09.269920 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" event={"ID":"3fa09ac0-300c-4d25-baa5-e07c192168e8","Type":"ContainerStarted","Data":"238cb550d548414b47afdd42258b32f0178423041bb1ecc080914eef3d75db95"} Jul 09 04:12:09.271687 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:09.271661 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-6q5mp" event={"ID":"e539f6ff-e687-4517-b313-96a208ec1d48","Type":"ContainerStarted","Data":"6352ae408bce0d30366e8b1bd633d252ec34a544c5fadff6a7258dc445df21b1"} Jul 09 04:12:09.271834 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:09.271818 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-6q5mp" Jul 09 04:12:09.273101 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:09.273073 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vljg2" event={"ID":"0a316c28-7d81-4e36-bff6-7ea5ef5ab248","Type":"ContainerStarted","Data":"16b4d0c3705bb09ea34d65726550fecf07a50b37de6ea305d9f5d962af753d50"} Jul 09 04:12:09.274476 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:09.274452 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4" event={"ID":"742d0176-a6e5-4a5a-932b-460367548241","Type":"ContainerStarted","Data":"c40ca2656c12c02f4e37bf9e29207d7ab3dec112aef6f827248c2ad22138c29e"} Jul 09 04:12:09.288815 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:09.288794 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-6q5mp" Jul 09 04:12:09.291477 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:09.291438 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-6q5mp" podStartSLOduration=1.76887247 podStartE2EDuration="21.291420372s" totalImagesPullingTime="19.522547902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:11:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:12:09.290194289 +0000 UTC m=+319.513812779" watchObservedRunningTime="2026-07-09 04:12:09.291420372 +0000 UTC m=+319.515038861" Jul 09 04:12:10.573818 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:10.573186 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cd88777bb-gm7kj"] Jul 09 04:12:14.300924 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:14.300880 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4" event={"ID":"742d0176-a6e5-4a5a-932b-460367548241","Type":"ContainerStarted","Data":"0b069c6ec64789d01daebefdc86936c8fede9d23a9235969fa644989885de50e"} Jul 09 04:12:14.301461 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:14.300958 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4" event={"ID":"742d0176-a6e5-4a5a-932b-460367548241","Type":"ContainerStarted","Data":"30e1d78246f4917f7eeb73c62cb67c549d4335821a10a75e1b94a76c764e94e6"} Jul 09 04:12:14.301461 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:14.300991 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4" event={"ID":"742d0176-a6e5-4a5a-932b-460367548241","Type":"ContainerStarted","Data":"0a5d3c27b28611192f19bbde7cb7296fb47dcf67af05d6f64bcc8e0d718d73ca"} Jul 09 04:12:14.303473 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:14.303444 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xrj2l" event={"ID":"52f790aa-4d62-4d1f-afee-2fb4aa845265","Type":"ContainerStarted","Data":"491b46ee1af38892af9fae29248272f4acff28647d572c61f6543cde6263baad"} Jul 09 04:12:14.305179 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:14.305153 2534 generic.go:356] "Generic (PLEG): container finished" podID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerID="a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881" exitCode=0 Jul 09 04:12:14.305320 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:14.305242 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2433376d-b2a6-424d-a06a-bf39dbb88b72","Type":"ContainerDied","Data":"a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881"} Jul 09 04:12:14.308097 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:14.308070 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" event={"ID":"3fa09ac0-300c-4d25-baa5-e07c192168e8","Type":"ContainerStarted","Data":"f73f5ea0143ce22ff953d1b1e1879e9c6d626893cc2b7e165c58db729eadc168"} Jul 09 04:12:14.308216 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:14.308140 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" event={"ID":"3fa09ac0-300c-4d25-baa5-e07c192168e8","Type":"ContainerStarted","Data":"ccb4b2a8b83d29c7a684a6e8bbe2018ad92f2ca8b430dd171930871c553a6061"} Jul 09 04:12:14.308216 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:14.308180 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" event={"ID":"3fa09ac0-300c-4d25-baa5-e07c192168e8","Type":"ContainerStarted","Data":"10e0062a812ddc9a457c781b6c4f2b1fc8a50175c712304f3781008a627c2213"} Jul 09 04:12:14.309864 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:14.309842 2534 generic.go:356] "Generic (PLEG): container finished" podID="0a316c28-7d81-4e36-bff6-7ea5ef5ab248" containerID="e0bfb2b9e6959810945e263a62924975ec190c0095a4045f1898386a4bdae1e9" exitCode=0 Jul 09 04:12:14.309949 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:14.309874 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vljg2" event={"ID":"0a316c28-7d81-4e36-bff6-7ea5ef5ab248","Type":"ContainerDied","Data":"e0bfb2b9e6959810945e263a62924975ec190c0095a4045f1898386a4bdae1e9"} Jul 09 04:12:14.319181 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:14.319128 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4cnc4" podStartSLOduration=7.663432739 podStartE2EDuration="12.319110361s" totalImagesPullingTime="4.655677622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:12:14.317962424 +0000 UTC m=+324.541580909" watchObservedRunningTime="2026-07-09 04:12:14.319110361 +0000 UTC m=+324.542728852" Jul 09 04:12:14.352049 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:14.351969 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-xrj2l" podStartSLOduration=7.917706578 podStartE2EDuration="12.351948231s" totalImagesPullingTime="4.434241653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:12:14.350549513 +0000 UTC m=+324.574168002" watchObservedRunningTime="2026-07-09 04:12:14.351948231 +0000 UTC m=+324.575566758" Jul 09 04:12:15.319489 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:15.319386 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vljg2" event={"ID":"0a316c28-7d81-4e36-bff6-7ea5ef5ab248","Type":"ContainerStarted","Data":"b7b01c9b8992857a2d13dffc260c8ea7a896c3c4de35cb457c05677694135c00"} Jul 09 04:12:15.320936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:15.320110 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vljg2" event={"ID":"0a316c28-7d81-4e36-bff6-7ea5ef5ab248","Type":"ContainerStarted","Data":"3c935ddb45bd25ca48128b2cf27301af57270bea4085131c267d400907fb87ec"} Jul 09 04:12:15.341774 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:15.341713 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vljg2" podStartSLOduration=8.38896997 podStartE2EDuration="13.341693318s" totalImagesPullingTime="4.817572905s" totalInitContainerRuntime="135.150443ms" isStatefulPod=true podCreationTimestamp="2026-07-09 04:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:12:15.339315921 +0000 UTC m=+325.562934410" watchObservedRunningTime="2026-07-09 04:12:15.341693318 +0000 UTC m=+325.565311807" Jul 09 04:12:16.328905 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:16.328796 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" event={"ID":"3fa09ac0-300c-4d25-baa5-e07c192168e8","Type":"ContainerStarted","Data":"67d4582fbfbed21425a5a542c3b4a4737eb9e1ad6ef08de2b4e60de8963e81a5"} Jul 09 04:12:16.328905 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:16.328877 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" event={"ID":"3fa09ac0-300c-4d25-baa5-e07c192168e8","Type":"ContainerStarted","Data":"56e0e7402616815b811cb174a01c7836de3d82ee3fac853e18d98363109dd2f7"} Jul 09 04:12:16.329582 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:16.328923 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" event={"ID":"3fa09ac0-300c-4d25-baa5-e07c192168e8","Type":"ContainerStarted","Data":"fee0a171703d250d517dcc3cc85d3812f3c7e330e45fa304f6ab105ca534613a"} Jul 09 04:12:16.330163 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:16.330142 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:16.352977 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:16.352899 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" podStartSLOduration=4.901023395 podStartE2EDuration="11.352875975s" totalImagesPullingTime="6.45185258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:12:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:12:16.350435829 +0000 UTC m=+326.574054349" watchObservedRunningTime="2026-07-09 04:12:16.352875975 +0000 UTC m=+326.576494464" Jul 09 04:12:19.345117 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:19.345088 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2433376d-b2a6-424d-a06a-bf39dbb88b72","Type":"ContainerStarted","Data":"ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3"} Jul 09 04:12:19.345483 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:19.345182 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2433376d-b2a6-424d-a06a-bf39dbb88b72","Type":"ContainerStarted","Data":"eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b"} Jul 09 04:12:19.345483 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:19.345218 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2433376d-b2a6-424d-a06a-bf39dbb88b72","Type":"ContainerStarted","Data":"3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd"} Jul 09 04:12:19.345483 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:19.345256 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2433376d-b2a6-424d-a06a-bf39dbb88b72","Type":"ContainerStarted","Data":"ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28"} Jul 09 04:12:20.352204 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:20.352171 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2433376d-b2a6-424d-a06a-bf39dbb88b72","Type":"ContainerStarted","Data":"d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71"} Jul 09 04:12:20.352689 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:20.352234 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2433376d-b2a6-424d-a06a-bf39dbb88b72","Type":"ContainerStarted","Data":"a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b"} Jul 09 04:12:20.378801 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:20.378714 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.341551117 podStartE2EDuration="12.378693293s" totalImagesPullingTime="8.988860738s" totalInitContainerRuntime="48.281438ms" isStatefulPod=false podCreationTimestamp="2026-07-09 04:12:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:12:20.377165265 +0000 UTC m=+330.600783758" watchObservedRunningTime="2026-07-09 04:12:20.378693293 +0000 UTC m=+330.602311783" Jul 09 04:12:22.339112 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:22.339082 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-8fd4cf69b-c4gtr" Jul 09 04:12:23.818819 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:23.818781 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:12:29.391747 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:29.391714 2534 generic.go:356] "Generic (PLEG): container finished" podID="900fd2e6-681b-451f-8f39-92d8d13bf9e5" containerID="a2168fc959b068da8a0ca38813fddd5c8f5a5cb231d956e1cfc3c87e6bd08fae" exitCode=0 Jul 09 04:12:29.392159 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:29.391757 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-8qmfm" event={"ID":"900fd2e6-681b-451f-8f39-92d8d13bf9e5","Type":"ContainerDied","Data":"a2168fc959b068da8a0ca38813fddd5c8f5a5cb231d956e1cfc3c87e6bd08fae"} Jul 09 04:12:29.392159 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:29.392072 2534 scope.go:117] "RemoveContainer" containerID="a2168fc959b068da8a0ca38813fddd5c8f5a5cb231d956e1cfc3c87e6bd08fae" Jul 09 04:12:30.280167 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:30.280137 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6prtw_675fded2-27b6-4bda-83ad-84f46e175548/dns/0.log" Jul 09 04:12:30.285642 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:30.285619 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6prtw_675fded2-27b6-4bda-83ad-84f46e175548/kube-rbac-proxy/0.log" Jul 09 04:12:30.397650 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:30.397618 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-8qmfm" event={"ID":"900fd2e6-681b-451f-8f39-92d8d13bf9e5","Type":"ContainerStarted","Data":"0fdaeff6065c3f31cf01285c48230398ea207a4a1315fb073f7b80ce1eb07fbc"} Jul 09 04:12:30.399099 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:30.399075 2534 generic.go:356] "Generic (PLEG): container finished" podID="0c2522b9-d54d-4091-907e-eb613d77b5c9" containerID="c96c66e1a0fa7fde51ddbb4a8bef32367c766abac052be4ab87d969c0bb82611" exitCode=0 Jul 09 04:12:30.399202 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:30.399150 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-2lxfd" event={"ID":"0c2522b9-d54d-4091-907e-eb613d77b5c9","Type":"ContainerDied","Data":"c96c66e1a0fa7fde51ddbb4a8bef32367c766abac052be4ab87d969c0bb82611"} Jul 09 04:12:30.399445 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:30.399432 2534 scope.go:117] "RemoveContainer" containerID="c96c66e1a0fa7fde51ddbb4a8bef32367c766abac052be4ab87d969c0bb82611" Jul 09 04:12:30.855068 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:30.855034 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zpr9q_8104c5b9-011c-4e10-9a15-53bdd52870f0/dns-node-resolver/0.log" Jul 09 04:12:31.405000 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:31.404965 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-2lxfd" event={"ID":"0c2522b9-d54d-4091-907e-eb613d77b5c9","Type":"ContainerStarted","Data":"e5afb1caffd895135a09dd960c9a62380161dc1187d67558b10856c200e22c98"} Jul 09 04:12:35.605037 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:35.604998 2534 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" podUID="8b49d1ca-6e2d-4701-b291-a1a9c9810431" containerName="registry" containerID="cri-o://6b750120dc81fc9d69bbecad754bd42fd972968a31d2de2670764c75de869f62" gracePeriod=30 Jul 09 04:12:35.867180 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:35.867122 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:12:35.937572 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:35.937542 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls\") pod \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " Jul 09 04:12:35.937704 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:35.937659 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mnrdf\" (UniqueName: \"kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-kube-api-access-mnrdf\") pod \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " Jul 09 04:12:35.937704 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:35.937691 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8b49d1ca-6e2d-4701-b291-a1a9c9810431-ca-trust-extracted\") pod \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " Jul 09 04:12:35.937820 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:35.937720 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-bound-sa-token\") pod \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " Jul 09 04:12:35.937820 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:35.937753 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8b49d1ca-6e2d-4701-b291-a1a9c9810431-installation-pull-secrets\") pod \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " Jul 09 04:12:35.937820 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:35.937778 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-certificates\") pod \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " Jul 09 04:12:35.937820 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:35.937809 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b49d1ca-6e2d-4701-b291-a1a9c9810431-trusted-ca\") pod \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " Jul 09 04:12:35.937997 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:35.937841 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8b49d1ca-6e2d-4701-b291-a1a9c9810431-image-registry-private-configuration\") pod \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\" (UID: \"8b49d1ca-6e2d-4701-b291-a1a9c9810431\") " Jul 09 04:12:35.938481 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:35.938214 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "8b49d1ca-6e2d-4701-b291-a1a9c9810431" (UID: "8b49d1ca-6e2d-4701-b291-a1a9c9810431"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:12:35.938717 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:35.938668 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b49d1ca-6e2d-4701-b291-a1a9c9810431-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "8b49d1ca-6e2d-4701-b291-a1a9c9810431" (UID: "8b49d1ca-6e2d-4701-b291-a1a9c9810431"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:12:35.940469 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:35.940443 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b49d1ca-6e2d-4701-b291-a1a9c9810431-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "8b49d1ca-6e2d-4701-b291-a1a9c9810431" (UID: "8b49d1ca-6e2d-4701-b291-a1a9c9810431"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:12:35.940565 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:35.940475 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8b49d1ca-6e2d-4701-b291-a1a9c9810431" (UID: "8b49d1ca-6e2d-4701-b291-a1a9c9810431"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:12:35.940565 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:35.940551 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b49d1ca-6e2d-4701-b291-a1a9c9810431-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "8b49d1ca-6e2d-4701-b291-a1a9c9810431" (UID: "8b49d1ca-6e2d-4701-b291-a1a9c9810431"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:12:35.940670 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:35.940614 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-kube-api-access-mnrdf" (OuterVolumeSpecName: "kube-api-access-mnrdf") pod "8b49d1ca-6e2d-4701-b291-a1a9c9810431" (UID: "8b49d1ca-6e2d-4701-b291-a1a9c9810431"). InnerVolumeSpecName "kube-api-access-mnrdf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:12:35.940670 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:35.940636 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "8b49d1ca-6e2d-4701-b291-a1a9c9810431" (UID: "8b49d1ca-6e2d-4701-b291-a1a9c9810431"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:12:35.946059 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:35.946031 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b49d1ca-6e2d-4701-b291-a1a9c9810431-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "8b49d1ca-6e2d-4701-b291-a1a9c9810431" (UID: "8b49d1ca-6e2d-4701-b291-a1a9c9810431"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:12:36.038894 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:36.038868 2534 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b49d1ca-6e2d-4701-b291-a1a9c9810431-trusted-ca\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:12:36.038894 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:36.038890 2534 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8b49d1ca-6e2d-4701-b291-a1a9c9810431-image-registry-private-configuration\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:12:36.038894 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:36.038900 2534 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-tls\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:12:36.039081 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:36.038909 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnrdf\" (UniqueName: \"kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-kube-api-access-mnrdf\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:12:36.039081 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:36.038918 2534 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8b49d1ca-6e2d-4701-b291-a1a9c9810431-ca-trust-extracted\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:12:36.039081 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:36.038926 2534 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b49d1ca-6e2d-4701-b291-a1a9c9810431-bound-sa-token\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:12:36.039081 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:36.038935 2534 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8b49d1ca-6e2d-4701-b291-a1a9c9810431-installation-pull-secrets\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:12:36.039081 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:36.038942 2534 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8b49d1ca-6e2d-4701-b291-a1a9c9810431-registry-certificates\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:12:36.425623 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:36.425586 2534 generic.go:356] "Generic (PLEG): container finished" podID="8b49d1ca-6e2d-4701-b291-a1a9c9810431" containerID="6b750120dc81fc9d69bbecad754bd42fd972968a31d2de2670764c75de869f62" exitCode=0 Jul 09 04:12:36.425801 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:36.425632 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" Jul 09 04:12:36.425801 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:36.425634 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" event={"ID":"8b49d1ca-6e2d-4701-b291-a1a9c9810431","Type":"ContainerDied","Data":"6b750120dc81fc9d69bbecad754bd42fd972968a31d2de2670764c75de869f62"} Jul 09 04:12:36.425801 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:36.425753 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cd88777bb-gm7kj" event={"ID":"8b49d1ca-6e2d-4701-b291-a1a9c9810431","Type":"ContainerDied","Data":"e077245447f41eedf3ce98dd2b681047d8c59a80f2090d34bea65baf5b727697"} Jul 09 04:12:36.425801 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:36.425783 2534 scope.go:117] "RemoveContainer" containerID="6b750120dc81fc9d69bbecad754bd42fd972968a31d2de2670764c75de869f62" Jul 09 04:12:36.434157 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:36.434144 2534 scope.go:117] "RemoveContainer" containerID="6b750120dc81fc9d69bbecad754bd42fd972968a31d2de2670764c75de869f62" Jul 09 04:12:36.434456 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:12:36.434427 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b750120dc81fc9d69bbecad754bd42fd972968a31d2de2670764c75de869f62\": container with ID starting with 6b750120dc81fc9d69bbecad754bd42fd972968a31d2de2670764c75de869f62 not found: ID does not exist" containerID="6b750120dc81fc9d69bbecad754bd42fd972968a31d2de2670764c75de869f62" Jul 09 04:12:36.434502 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:36.434464 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b750120dc81fc9d69bbecad754bd42fd972968a31d2de2670764c75de869f62"} err="failed to get container status \"6b750120dc81fc9d69bbecad754bd42fd972968a31d2de2670764c75de869f62\": rpc error: code = NotFound desc = could not find container \"6b750120dc81fc9d69bbecad754bd42fd972968a31d2de2670764c75de869f62\": container with ID starting with 6b750120dc81fc9d69bbecad754bd42fd972968a31d2de2670764c75de869f62 not found: ID does not exist" Jul 09 04:12:36.442105 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:36.442081 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cd88777bb-gm7kj"] Jul 09 04:12:36.445711 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:36.445685 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6cd88777bb-gm7kj"] Jul 09 04:12:38.249948 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:38.249917 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b49d1ca-6e2d-4701-b291-a1a9c9810431" path="/var/lib/kubelet/pods/8b49d1ca-6e2d-4701-b291-a1a9c9810431/volumes" Jul 09 04:12:40.445885 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:40.445856 2534 generic.go:356] "Generic (PLEG): container finished" podID="8872a475-7573-4057-a22c-236c5e337a1a" containerID="b181615c6acf5f89003ba97cc61dbf98d4b04996fdea725590d9ca958a270cc6" exitCode=0 Jul 09 04:12:40.446294 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:40.445936 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w48tg" event={"ID":"8872a475-7573-4057-a22c-236c5e337a1a","Type":"ContainerDied","Data":"b181615c6acf5f89003ba97cc61dbf98d4b04996fdea725590d9ca958a270cc6"} Jul 09 04:12:40.446294 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:40.446232 2534 scope.go:117] "RemoveContainer" containerID="b181615c6acf5f89003ba97cc61dbf98d4b04996fdea725590d9ca958a270cc6" Jul 09 04:12:41.452145 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:12:41.452064 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w48tg" event={"ID":"8872a475-7573-4057-a22c-236c5e337a1a","Type":"ContainerStarted","Data":"b2e3a5232229ba381b0a4c1f1d8fcd52da6f8e7372f4dafdd4a31f4bea2685f4"} Jul 09 04:13:04.282446 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:04.282396 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs\") pod \"network-metrics-daemon-b4gcd\" (UID: \"9fefcefc-4128-4422-98de-b17e5949cfc7\") " pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:13:04.284956 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:04.284931 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fefcefc-4128-4422-98de-b17e5949cfc7-metrics-certs\") pod \"network-metrics-daemon-b4gcd\" (UID: \"9fefcefc-4128-4422-98de-b17e5949cfc7\") " pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:13:04.447658 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:04.447628 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v2z2w\"" Jul 09 04:13:04.455522 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:04.455501 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4gcd" Jul 09 04:13:04.590751 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:04.590728 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b4gcd"] Jul 09 04:13:04.593241 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:13:04.593219 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fefcefc_4128_4422_98de_b17e5949cfc7.slice/crio-f53f5455daadc4bf88117da3b3fe8eecfc028b34c87c735308f6d023982736df WatchSource:0}: Error finding container f53f5455daadc4bf88117da3b3fe8eecfc028b34c87c735308f6d023982736df: Status 404 returned error can't find the container with id f53f5455daadc4bf88117da3b3fe8eecfc028b34c87c735308f6d023982736df Jul 09 04:13:05.558907 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:05.558863 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b4gcd" event={"ID":"9fefcefc-4128-4422-98de-b17e5949cfc7","Type":"ContainerStarted","Data":"f53f5455daadc4bf88117da3b3fe8eecfc028b34c87c735308f6d023982736df"} Jul 09 04:13:06.570984 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:06.570943 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b4gcd" event={"ID":"9fefcefc-4128-4422-98de-b17e5949cfc7","Type":"ContainerStarted","Data":"9f3a181950c1d0049d1898fd3dd94be999359606e484be768a989d16d3daa9b5"} Jul 09 04:13:06.571417 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:06.571023 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b4gcd" event={"ID":"9fefcefc-4128-4422-98de-b17e5949cfc7","Type":"ContainerStarted","Data":"a8b516071cddaa0aec6343de09d48c33bc425182518938bef8d7050e0c813caa"} Jul 09 04:13:06.587891 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:06.587845 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-b4gcd" podStartSLOduration=374.488290473 podStartE2EDuration="6m15.58783068s" totalImagesPullingTime="1.099540207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:06:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:13:06.586189039 +0000 UTC m=+376.809807526" watchObservedRunningTime="2026-07-09 04:13:06.58783068 +0000 UTC m=+376.811449197" Jul 09 04:13:08.819911 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:08.819869 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:08.840129 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:08.840101 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:09.599817 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:09.599783 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:26.862509 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:26.862474 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 04:13:26.863524 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:26.863462 2534 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="prometheus" containerID="cri-o://ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28" gracePeriod=600 Jul 09 04:13:26.863880 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:26.863854 2534 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="kube-rbac-proxy-thanos" containerID="cri-o://d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71" gracePeriod=600 Jul 09 04:13:26.863998 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:26.863968 2534 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="kube-rbac-proxy" containerID="cri-o://a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b" gracePeriod=600 Jul 09 04:13:26.864069 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:26.864047 2534 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="kube-rbac-proxy-web" containerID="cri-o://ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3" gracePeriod=600 Jul 09 04:13:26.864127 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:26.864112 2534 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="thanos-sidecar" containerID="cri-o://eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b" gracePeriod=600 Jul 09 04:13:26.864201 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:26.864176 2534 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="config-reloader" containerID="cri-o://3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd" gracePeriod=600 Jul 09 04:13:27.129883 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.129859 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:27.294441 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.294411 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-k8s-rulefiles-0\") pod \"2433376d-b2a6-424d-a06a-bf39dbb88b72\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " Jul 09 04:13:27.294622 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.294453 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-k8s-rulefiles-2\") pod \"2433376d-b2a6-424d-a06a-bf39dbb88b72\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " Jul 09 04:13:27.294622 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.294476 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"2433376d-b2a6-424d-a06a-bf39dbb88b72\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " Jul 09 04:13:27.294622 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.294494 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-web-config\") pod \"2433376d-b2a6-424d-a06a-bf39dbb88b72\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " Jul 09 04:13:27.294622 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.294515 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-trusted-ca-bundle\") pod \"2433376d-b2a6-424d-a06a-bf39dbb88b72\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " Jul 09 04:13:27.294622 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.294541 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-k8s-rulefiles-1\") pod \"2433376d-b2a6-424d-a06a-bf39dbb88b72\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " Jul 09 04:13:27.294622 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.294565 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"2433376d-b2a6-424d-a06a-bf39dbb88b72\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " Jul 09 04:13:27.294622 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.294594 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-grpc-tls\") pod \"2433376d-b2a6-424d-a06a-bf39dbb88b72\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " Jul 09 04:13:27.294622 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.294618 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2433376d-b2a6-424d-a06a-bf39dbb88b72-config-out\") pod \"2433376d-b2a6-424d-a06a-bf39dbb88b72\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " Jul 09 04:13:27.295004 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.294639 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2433376d-b2a6-424d-a06a-bf39dbb88b72-tls-assets\") pod \"2433376d-b2a6-424d-a06a-bf39dbb88b72\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " Jul 09 04:13:27.295004 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.294676 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-kube-rbac-proxy\") pod \"2433376d-b2a6-424d-a06a-bf39dbb88b72\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " Jul 09 04:13:27.295004 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.294700 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-configmap-serving-certs-ca-bundle\") pod \"2433376d-b2a6-424d-a06a-bf39dbb88b72\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " Jul 09 04:13:27.295004 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.294728 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-configmap-metrics-client-ca\") pod \"2433376d-b2a6-424d-a06a-bf39dbb88b72\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " Jul 09 04:13:27.295004 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.294754 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-k8s-db\") pod \"2433376d-b2a6-424d-a06a-bf39dbb88b72\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " Jul 09 04:13:27.295004 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.294781 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-config\") pod \"2433376d-b2a6-424d-a06a-bf39dbb88b72\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " Jul 09 04:13:27.295004 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.294823 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-thanos-prometheus-http-client-file\") pod \"2433376d-b2a6-424d-a06a-bf39dbb88b72\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " Jul 09 04:13:27.295004 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.294841 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "2433376d-b2a6-424d-a06a-bf39dbb88b72" (UID: "2433376d-b2a6-424d-a06a-bf39dbb88b72"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:13:27.295004 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.294860 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-metrics-client-certs\") pod \"2433376d-b2a6-424d-a06a-bf39dbb88b72\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " Jul 09 04:13:27.295004 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.294887 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-prometheus-k8s-tls\") pod \"2433376d-b2a6-424d-a06a-bf39dbb88b72\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " Jul 09 04:13:27.295004 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.294913 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2d8xw\" (UniqueName: \"kubernetes.io/projected/2433376d-b2a6-424d-a06a-bf39dbb88b72-kube-api-access-2d8xw\") pod \"2433376d-b2a6-424d-a06a-bf39dbb88b72\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " Jul 09 04:13:27.295004 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.294909 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "2433376d-b2a6-424d-a06a-bf39dbb88b72" (UID: "2433376d-b2a6-424d-a06a-bf39dbb88b72"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:13:27.295004 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.294999 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "2433376d-b2a6-424d-a06a-bf39dbb88b72" (UID: "2433376d-b2a6-424d-a06a-bf39dbb88b72"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:13:27.295651 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.295005 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-configmap-kubelet-serving-ca-bundle\") pod \"2433376d-b2a6-424d-a06a-bf39dbb88b72\" (UID: \"2433376d-b2a6-424d-a06a-bf39dbb88b72\") " Jul 09 04:13:27.295651 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.295373 2534 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:13:27.295651 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.295394 2534 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:13:27.295651 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.295412 2534 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:13:27.295651 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.295418 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "2433376d-b2a6-424d-a06a-bf39dbb88b72" (UID: "2433376d-b2a6-424d-a06a-bf39dbb88b72"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:13:27.295651 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.295585 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "2433376d-b2a6-424d-a06a-bf39dbb88b72" (UID: "2433376d-b2a6-424d-a06a-bf39dbb88b72"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:13:27.296199 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.296174 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "2433376d-b2a6-424d-a06a-bf39dbb88b72" (UID: "2433376d-b2a6-424d-a06a-bf39dbb88b72"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:13:27.297015 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.296631 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "2433376d-b2a6-424d-a06a-bf39dbb88b72" (UID: "2433376d-b2a6-424d-a06a-bf39dbb88b72"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:13:27.297917 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.297890 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "2433376d-b2a6-424d-a06a-bf39dbb88b72" (UID: "2433376d-b2a6-424d-a06a-bf39dbb88b72"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:13:27.298166 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.298145 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "2433376d-b2a6-424d-a06a-bf39dbb88b72" (UID: "2433376d-b2a6-424d-a06a-bf39dbb88b72"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:13:27.298299 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.298220 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2433376d-b2a6-424d-a06a-bf39dbb88b72-kube-api-access-2d8xw" (OuterVolumeSpecName: "kube-api-access-2d8xw") pod "2433376d-b2a6-424d-a06a-bf39dbb88b72" (UID: "2433376d-b2a6-424d-a06a-bf39dbb88b72"). InnerVolumeSpecName "kube-api-access-2d8xw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:13:27.299008 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.298981 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "2433376d-b2a6-424d-a06a-bf39dbb88b72" (UID: "2433376d-b2a6-424d-a06a-bf39dbb88b72"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:13:27.299146 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.299107 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "2433376d-b2a6-424d-a06a-bf39dbb88b72" (UID: "2433376d-b2a6-424d-a06a-bf39dbb88b72"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:13:27.299452 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.299433 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2433376d-b2a6-424d-a06a-bf39dbb88b72-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "2433376d-b2a6-424d-a06a-bf39dbb88b72" (UID: "2433376d-b2a6-424d-a06a-bf39dbb88b72"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:13:27.299553 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.299518 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-config" (OuterVolumeSpecName: "config") pod "2433376d-b2a6-424d-a06a-bf39dbb88b72" (UID: "2433376d-b2a6-424d-a06a-bf39dbb88b72"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:13:27.299553 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.299544 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "2433376d-b2a6-424d-a06a-bf39dbb88b72" (UID: "2433376d-b2a6-424d-a06a-bf39dbb88b72"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:13:27.300712 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.300682 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2433376d-b2a6-424d-a06a-bf39dbb88b72-config-out" (OuterVolumeSpecName: "config-out") pod "2433376d-b2a6-424d-a06a-bf39dbb88b72" (UID: "2433376d-b2a6-424d-a06a-bf39dbb88b72"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:13:27.300775 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.300752 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "2433376d-b2a6-424d-a06a-bf39dbb88b72" (UID: "2433376d-b2a6-424d-a06a-bf39dbb88b72"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:13:27.300775 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.300761 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "2433376d-b2a6-424d-a06a-bf39dbb88b72" (UID: "2433376d-b2a6-424d-a06a-bf39dbb88b72"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:13:27.301092 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.301077 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "2433376d-b2a6-424d-a06a-bf39dbb88b72" (UID: "2433376d-b2a6-424d-a06a-bf39dbb88b72"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:13:27.309378 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.309360 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-web-config" (OuterVolumeSpecName: "web-config") pod "2433376d-b2a6-424d-a06a-bf39dbb88b72" (UID: "2433376d-b2a6-424d-a06a-bf39dbb88b72"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:13:27.396235 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.396176 2534 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-kube-rbac-proxy\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:13:27.396235 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.396194 2534 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:13:27.396235 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.396204 2534 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-configmap-metrics-client-ca\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:13:27.396235 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.396213 2534 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-k8s-db\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:13:27.396235 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.396234 2534 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-config\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:13:27.396451 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.396243 2534 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:13:27.396451 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.396251 2534 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-metrics-client-certs\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:13:27.396451 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.396260 2534 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:13:27.396451 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.396268 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2d8xw\" (UniqueName: \"kubernetes.io/projected/2433376d-b2a6-424d-a06a-bf39dbb88b72-kube-api-access-2d8xw\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:13:27.396451 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.396277 2534 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:13:27.396451 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.396308 2534 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2433376d-b2a6-424d-a06a-bf39dbb88b72-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:13:27.396451 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.396321 2534 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:13:27.396451 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.396334 2534 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-web-config\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:13:27.396451 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.396343 2534 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:13:27.396451 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.396351 2534 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2433376d-b2a6-424d-a06a-bf39dbb88b72-secret-grpc-tls\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:13:27.396451 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.396360 2534 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2433376d-b2a6-424d-a06a-bf39dbb88b72-config-out\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:13:27.396451 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.396366 2534 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2433376d-b2a6-424d-a06a-bf39dbb88b72-tls-assets\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:13:27.663918 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.663841 2534 generic.go:356] "Generic (PLEG): container finished" podID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerID="d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71" exitCode=0 Jul 09 04:13:27.663918 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.663869 2534 generic.go:356] "Generic (PLEG): container finished" podID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerID="a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b" exitCode=0 Jul 09 04:13:27.663918 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.663879 2534 generic.go:356] "Generic (PLEG): container finished" podID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerID="ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3" exitCode=0 Jul 09 04:13:27.663918 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.663889 2534 generic.go:356] "Generic (PLEG): container finished" podID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerID="eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b" exitCode=0 Jul 09 04:13:27.663918 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.663896 2534 generic.go:356] "Generic (PLEG): container finished" podID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerID="3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd" exitCode=0 Jul 09 04:13:27.663918 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.663900 2534 generic.go:356] "Generic (PLEG): container finished" podID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerID="ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28" exitCode=0 Jul 09 04:13:27.663918 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.663899 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2433376d-b2a6-424d-a06a-bf39dbb88b72","Type":"ContainerDied","Data":"d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71"} Jul 09 04:13:27.664237 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.663926 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:27.664237 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.663976 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2433376d-b2a6-424d-a06a-bf39dbb88b72","Type":"ContainerDied","Data":"a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b"} Jul 09 04:13:27.664237 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.664008 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2433376d-b2a6-424d-a06a-bf39dbb88b72","Type":"ContainerDied","Data":"ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3"} Jul 09 04:13:27.664237 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.664044 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2433376d-b2a6-424d-a06a-bf39dbb88b72","Type":"ContainerDied","Data":"eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b"} Jul 09 04:13:27.664237 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.664072 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2433376d-b2a6-424d-a06a-bf39dbb88b72","Type":"ContainerDied","Data":"3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd"} Jul 09 04:13:27.664237 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.664096 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2433376d-b2a6-424d-a06a-bf39dbb88b72","Type":"ContainerDied","Data":"ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28"} Jul 09 04:13:27.664237 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.664047 2534 scope.go:117] "RemoveContainer" containerID="d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71" Jul 09 04:13:27.664237 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.664129 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2433376d-b2a6-424d-a06a-bf39dbb88b72","Type":"ContainerDied","Data":"a88bc29d177d6e2dabccef7c2e673dd176876d945cb731c2d13bd7159cc7ea34"} Jul 09 04:13:27.673179 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.673156 2534 scope.go:117] "RemoveContainer" containerID="a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b" Jul 09 04:13:27.682063 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.682043 2534 scope.go:117] "RemoveContainer" containerID="ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3" Jul 09 04:13:27.690928 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.690906 2534 scope.go:117] "RemoveContainer" containerID="eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b" Jul 09 04:13:27.696403 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.696352 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 04:13:27.700032 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.700015 2534 scope.go:117] "RemoveContainer" containerID="3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd" Jul 09 04:13:27.709086 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.709054 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 04:13:27.709430 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.709393 2534 scope.go:117] "RemoveContainer" containerID="ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28" Jul 09 04:13:27.718096 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.718074 2534 scope.go:117] "RemoveContainer" containerID="a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881" Jul 09 04:13:27.725588 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.725557 2534 scope.go:117] "RemoveContainer" containerID="d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71" Jul 09 04:13:27.725832 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:13:27.725814 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71\": container with ID starting with d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71 not found: ID does not exist" containerID="d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71" Jul 09 04:13:27.725883 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.725841 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71"} err="failed to get container status \"d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71\": rpc error: code = NotFound desc = could not find container \"d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71\": container with ID starting with d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71 not found: ID does not exist" Jul 09 04:13:27.725883 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.725861 2534 scope.go:117] "RemoveContainer" containerID="a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b" Jul 09 04:13:27.726088 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:13:27.726073 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b\": container with ID starting with a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b not found: ID does not exist" containerID="a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b" Jul 09 04:13:27.726129 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.726092 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b"} err="failed to get container status \"a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b\": rpc error: code = NotFound desc = could not find container \"a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b\": container with ID starting with a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b not found: ID does not exist" Jul 09 04:13:27.726129 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.726104 2534 scope.go:117] "RemoveContainer" containerID="ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3" Jul 09 04:13:27.726328 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:13:27.726305 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3\": container with ID starting with ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3 not found: ID does not exist" containerID="ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3" Jul 09 04:13:27.726406 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.726330 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3"} err="failed to get container status \"ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3\": rpc error: code = NotFound desc = could not find container \"ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3\": container with ID starting with ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3 not found: ID does not exist" Jul 09 04:13:27.726406 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.726348 2534 scope.go:117] "RemoveContainer" containerID="eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b" Jul 09 04:13:27.726601 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:13:27.726582 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b\": container with ID starting with eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b not found: ID does not exist" containerID="eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b" Jul 09 04:13:27.726644 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.726604 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b"} err="failed to get container status \"eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b\": rpc error: code = NotFound desc = could not find container \"eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b\": container with ID starting with eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b not found: ID does not exist" Jul 09 04:13:27.726644 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.726616 2534 scope.go:117] "RemoveContainer" containerID="3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd" Jul 09 04:13:27.726849 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:13:27.726833 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd\": container with ID starting with 3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd not found: ID does not exist" containerID="3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd" Jul 09 04:13:27.726900 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.726855 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd"} err="failed to get container status \"3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd\": rpc error: code = NotFound desc = could not find container \"3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd\": container with ID starting with 3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd not found: ID does not exist" Jul 09 04:13:27.726900 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.726870 2534 scope.go:117] "RemoveContainer" containerID="ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28" Jul 09 04:13:27.727070 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:13:27.727051 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28\": container with ID starting with ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28 not found: ID does not exist" containerID="ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28" Jul 09 04:13:27.727133 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.727081 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28"} err="failed to get container status \"ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28\": rpc error: code = NotFound desc = could not find container \"ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28\": container with ID starting with ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28 not found: ID does not exist" Jul 09 04:13:27.727133 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.727096 2534 scope.go:117] "RemoveContainer" containerID="a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881" Jul 09 04:13:27.727375 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:13:27.727360 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881\": container with ID starting with a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881 not found: ID does not exist" containerID="a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881" Jul 09 04:13:27.727440 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.727376 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881"} err="failed to get container status \"a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881\": rpc error: code = NotFound desc = could not find container \"a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881\": container with ID starting with a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881 not found: ID does not exist" Jul 09 04:13:27.727440 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.727387 2534 scope.go:117] "RemoveContainer" containerID="d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71" Jul 09 04:13:27.727586 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.727566 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71"} err="failed to get container status \"d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71\": rpc error: code = NotFound desc = could not find container \"d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71\": container with ID starting with d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71 not found: ID does not exist" Jul 09 04:13:27.727636 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.727586 2534 scope.go:117] "RemoveContainer" containerID="a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b" Jul 09 04:13:27.727787 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.727770 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b"} err="failed to get container status \"a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b\": rpc error: code = NotFound desc = could not find container \"a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b\": container with ID starting with a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b not found: ID does not exist" Jul 09 04:13:27.727834 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.727789 2534 scope.go:117] "RemoveContainer" containerID="ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3" Jul 09 04:13:27.728074 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.728046 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3"} err="failed to get container status \"ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3\": rpc error: code = NotFound desc = could not find container \"ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3\": container with ID starting with ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3 not found: ID does not exist" Jul 09 04:13:27.728074 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.728074 2534 scope.go:117] "RemoveContainer" containerID="eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b" Jul 09 04:13:27.728377 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.728356 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b"} err="failed to get container status \"eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b\": rpc error: code = NotFound desc = could not find container \"eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b\": container with ID starting with eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b not found: ID does not exist" Jul 09 04:13:27.728461 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.728379 2534 scope.go:117] "RemoveContainer" containerID="3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd" Jul 09 04:13:27.728628 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.728615 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd"} err="failed to get container status \"3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd\": rpc error: code = NotFound desc = could not find container \"3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd\": container with ID starting with 3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd not found: ID does not exist" Jul 09 04:13:27.728680 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.728629 2534 scope.go:117] "RemoveContainer" containerID="ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28" Jul 09 04:13:27.728857 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.728838 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28"} err="failed to get container status \"ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28\": rpc error: code = NotFound desc = could not find container \"ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28\": container with ID starting with ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28 not found: ID does not exist" Jul 09 04:13:27.728907 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.728857 2534 scope.go:117] "RemoveContainer" containerID="a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881" Jul 09 04:13:27.729057 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.729042 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881"} err="failed to get container status \"a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881\": rpc error: code = NotFound desc = could not find container \"a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881\": container with ID starting with a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881 not found: ID does not exist" Jul 09 04:13:27.729104 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.729056 2534 scope.go:117] "RemoveContainer" containerID="d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71" Jul 09 04:13:27.729269 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.729254 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71"} err="failed to get container status \"d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71\": rpc error: code = NotFound desc = could not find container \"d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71\": container with ID starting with d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71 not found: ID does not exist" Jul 09 04:13:27.729343 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.729269 2534 scope.go:117] "RemoveContainer" containerID="a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b" Jul 09 04:13:27.729489 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.729474 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b"} err="failed to get container status \"a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b\": rpc error: code = NotFound desc = could not find container \"a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b\": container with ID starting with a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b not found: ID does not exist" Jul 09 04:13:27.729535 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.729488 2534 scope.go:117] "RemoveContainer" containerID="ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3" Jul 09 04:13:27.729689 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.729672 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3"} err="failed to get container status \"ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3\": rpc error: code = NotFound desc = could not find container \"ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3\": container with ID starting with ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3 not found: ID does not exist" Jul 09 04:13:27.729727 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.729690 2534 scope.go:117] "RemoveContainer" containerID="eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b" Jul 09 04:13:27.729924 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.729907 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b"} err="failed to get container status \"eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b\": rpc error: code = NotFound desc = could not find container \"eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b\": container with ID starting with eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b not found: ID does not exist" Jul 09 04:13:27.729924 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.729924 2534 scope.go:117] "RemoveContainer" containerID="3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd" Jul 09 04:13:27.730108 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.730093 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd"} err="failed to get container status \"3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd\": rpc error: code = NotFound desc = could not find container \"3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd\": container with ID starting with 3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd not found: ID does not exist" Jul 09 04:13:27.730149 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.730110 2534 scope.go:117] "RemoveContainer" containerID="ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28" Jul 09 04:13:27.730320 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.730304 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28"} err="failed to get container status \"ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28\": rpc error: code = NotFound desc = could not find container \"ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28\": container with ID starting with ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28 not found: ID does not exist" Jul 09 04:13:27.730376 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.730320 2534 scope.go:117] "RemoveContainer" containerID="a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881" Jul 09 04:13:27.730545 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.730526 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881"} err="failed to get container status \"a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881\": rpc error: code = NotFound desc = could not find container \"a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881\": container with ID starting with a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881 not found: ID does not exist" Jul 09 04:13:27.730585 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.730547 2534 scope.go:117] "RemoveContainer" containerID="d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71" Jul 09 04:13:27.730735 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.730720 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71"} err="failed to get container status \"d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71\": rpc error: code = NotFound desc = could not find container \"d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71\": container with ID starting with d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71 not found: ID does not exist" Jul 09 04:13:27.730774 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.730738 2534 scope.go:117] "RemoveContainer" containerID="a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b" Jul 09 04:13:27.730934 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.730916 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b"} err="failed to get container status \"a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b\": rpc error: code = NotFound desc = could not find container \"a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b\": container with ID starting with a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b not found: ID does not exist" Jul 09 04:13:27.730989 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.730934 2534 scope.go:117] "RemoveContainer" containerID="ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3" Jul 09 04:13:27.731158 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.731139 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3"} err="failed to get container status \"ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3\": rpc error: code = NotFound desc = could not find container \"ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3\": container with ID starting with ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3 not found: ID does not exist" Jul 09 04:13:27.731229 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.731159 2534 scope.go:117] "RemoveContainer" containerID="eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b" Jul 09 04:13:27.731678 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.731473 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b"} err="failed to get container status \"eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b\": rpc error: code = NotFound desc = could not find container \"eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b\": container with ID starting with eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b not found: ID does not exist" Jul 09 04:13:27.731678 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.731546 2534 scope.go:117] "RemoveContainer" containerID="3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd" Jul 09 04:13:27.732262 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.732233 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd"} err="failed to get container status \"3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd\": rpc error: code = NotFound desc = could not find container \"3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd\": container with ID starting with 3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd not found: ID does not exist" Jul 09 04:13:27.732354 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.732323 2534 scope.go:117] "RemoveContainer" containerID="ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28" Jul 09 04:13:27.732663 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.732645 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28"} err="failed to get container status \"ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28\": rpc error: code = NotFound desc = could not find container \"ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28\": container with ID starting with ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28 not found: ID does not exist" Jul 09 04:13:27.732744 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.732665 2534 scope.go:117] "RemoveContainer" containerID="a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881" Jul 09 04:13:27.732932 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.732915 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881"} err="failed to get container status \"a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881\": rpc error: code = NotFound desc = could not find container \"a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881\": container with ID starting with a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881 not found: ID does not exist" Jul 09 04:13:27.733002 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.732933 2534 scope.go:117] "RemoveContainer" containerID="d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71" Jul 09 04:13:27.733169 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.733152 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71"} err="failed to get container status \"d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71\": rpc error: code = NotFound desc = could not find container \"d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71\": container with ID starting with d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71 not found: ID does not exist" Jul 09 04:13:27.733216 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.733170 2534 scope.go:117] "RemoveContainer" containerID="a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b" Jul 09 04:13:27.733444 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.733427 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b"} err="failed to get container status \"a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b\": rpc error: code = NotFound desc = could not find container \"a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b\": container with ID starting with a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b not found: ID does not exist" Jul 09 04:13:27.733515 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.733444 2534 scope.go:117] "RemoveContainer" containerID="ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3" Jul 09 04:13:27.733681 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.733662 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3"} err="failed to get container status \"ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3\": rpc error: code = NotFound desc = could not find container \"ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3\": container with ID starting with ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3 not found: ID does not exist" Jul 09 04:13:27.733734 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.733683 2534 scope.go:117] "RemoveContainer" containerID="eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b" Jul 09 04:13:27.733878 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.733864 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b"} err="failed to get container status \"eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b\": rpc error: code = NotFound desc = could not find container \"eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b\": container with ID starting with eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b not found: ID does not exist" Jul 09 04:13:27.733920 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.733878 2534 scope.go:117] "RemoveContainer" containerID="3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd" Jul 09 04:13:27.733990 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.733971 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 04:13:27.734093 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734074 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd"} err="failed to get container status \"3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd\": rpc error: code = NotFound desc = could not find container \"3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd\": container with ID starting with 3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd not found: ID does not exist" Jul 09 04:13:27.734160 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734094 2534 scope.go:117] "RemoveContainer" containerID="ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28" Jul 09 04:13:27.734357 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734339 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28"} err="failed to get container status \"ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28\": rpc error: code = NotFound desc = could not find container \"ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28\": container with ID starting with ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28 not found: ID does not exist" Jul 09 04:13:27.734430 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734357 2534 scope.go:117] "RemoveContainer" containerID="a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881" Jul 09 04:13:27.734430 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734372 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b49d1ca-6e2d-4701-b291-a1a9c9810431" containerName="registry" Jul 09 04:13:27.734430 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734384 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b49d1ca-6e2d-4701-b291-a1a9c9810431" containerName="registry" Jul 09 04:13:27.734430 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734392 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="kube-rbac-proxy-web" Jul 09 04:13:27.734430 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734397 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="kube-rbac-proxy-web" Jul 09 04:13:27.734430 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734408 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="config-reloader" Jul 09 04:13:27.734430 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734414 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="config-reloader" Jul 09 04:13:27.734430 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734424 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="kube-rbac-proxy" Jul 09 04:13:27.734430 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734429 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="kube-rbac-proxy" Jul 09 04:13:27.734730 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734440 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="prometheus" Jul 09 04:13:27.734730 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734445 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="prometheus" Jul 09 04:13:27.734730 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734452 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="kube-rbac-proxy-thanos" Jul 09 04:13:27.734730 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734457 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="kube-rbac-proxy-thanos" Jul 09 04:13:27.734730 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734470 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="thanos-sidecar" Jul 09 04:13:27.734730 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734475 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="thanos-sidecar" Jul 09 04:13:27.734730 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734483 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="init-config-reloader" Jul 09 04:13:27.734730 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734487 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="init-config-reloader" Jul 09 04:13:27.734730 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734535 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b49d1ca-6e2d-4701-b291-a1a9c9810431" containerName="registry" Jul 09 04:13:27.734730 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734542 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="kube-rbac-proxy" Jul 09 04:13:27.734730 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734548 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="kube-rbac-proxy-web" Jul 09 04:13:27.734730 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734555 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="thanos-sidecar" Jul 09 04:13:27.734730 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734561 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="config-reloader" Jul 09 04:13:27.734730 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734566 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="prometheus" Jul 09 04:13:27.734730 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734572 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" containerName="kube-rbac-proxy-thanos" Jul 09 04:13:27.734730 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734586 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881"} err="failed to get container status \"a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881\": rpc error: code = NotFound desc = could not find container \"a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881\": container with ID starting with a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881 not found: ID does not exist" Jul 09 04:13:27.734730 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734602 2534 scope.go:117] "RemoveContainer" containerID="d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71" Jul 09 04:13:27.735384 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734806 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71"} err="failed to get container status \"d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71\": rpc error: code = NotFound desc = could not find container \"d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71\": container with ID starting with d32b752e088320c039d58a47440c2e526778d974bfd88371e163629b75716d71 not found: ID does not exist" Jul 09 04:13:27.735384 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.734821 2534 scope.go:117] "RemoveContainer" containerID="a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b" Jul 09 04:13:27.735384 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.735055 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b"} err="failed to get container status \"a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b\": rpc error: code = NotFound desc = could not find container \"a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b\": container with ID starting with a035235136d72dc41f2b3da05c1032df8345415f1b75a6908b7a1a6da83aae8b not found: ID does not exist" Jul 09 04:13:27.735384 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.735083 2534 scope.go:117] "RemoveContainer" containerID="ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3" Jul 09 04:13:27.735530 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.735448 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3"} err="failed to get container status \"ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3\": rpc error: code = NotFound desc = could not find container \"ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3\": container with ID starting with ee52fe1e807c79a504ff64ec0ca1b67aa496e4ee411099fdc266cd109b98f4c3 not found: ID does not exist" Jul 09 04:13:27.735530 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.735471 2534 scope.go:117] "RemoveContainer" containerID="eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b" Jul 09 04:13:27.735770 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.735738 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b"} err="failed to get container status \"eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b\": rpc error: code = NotFound desc = could not find container \"eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b\": container with ID starting with eca3b9e5c2c91b4b634b7fa0caec66fc36e67c26d3a9ea78bcec1c482faa765b not found: ID does not exist" Jul 09 04:13:27.735854 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.735771 2534 scope.go:117] "RemoveContainer" containerID="3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd" Jul 09 04:13:27.736066 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.736034 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd"} err="failed to get container status \"3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd\": rpc error: code = NotFound desc = could not find container \"3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd\": container with ID starting with 3a504767a74e5e0b54e4c9d695144869210bc7f848dab4cd627784c1ab8547cd not found: ID does not exist" Jul 09 04:13:27.736066 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.736054 2534 scope.go:117] "RemoveContainer" containerID="ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28" Jul 09 04:13:27.736352 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.736335 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28"} err="failed to get container status \"ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28\": rpc error: code = NotFound desc = could not find container \"ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28\": container with ID starting with ad84c1de375a0413c0b0d0f66d465476837cb9a25e4da2002d4ce5785271cd28 not found: ID does not exist" Jul 09 04:13:27.736352 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.736353 2534 scope.go:117] "RemoveContainer" containerID="a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881" Jul 09 04:13:27.738189 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.738157 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881"} err="failed to get container status \"a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881\": rpc error: code = NotFound desc = could not find container \"a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881\": container with ID starting with a314481f459d479d15185b92c56eb7548a659feb02671a107d103d4c71030881 not found: ID does not exist" Jul 09 04:13:27.740292 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.740263 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:27.742878 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.742835 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 09 04:13:27.742878 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.742865 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 09 04:13:27.743066 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.742867 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 09 04:13:27.743066 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.742951 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 09 04:13:27.743066 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.742959 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 09 04:13:27.743266 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.743105 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 09 04:13:27.743266 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.743205 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 09 04:13:27.743455 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.743368 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 09 04:13:27.743455 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.743439 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 09 04:13:27.743559 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.743478 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 09 04:13:27.743888 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.743873 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 09 04:13:27.743962 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.743903 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-n24b7\"" Jul 09 04:13:27.743962 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.743923 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 09 04:13:27.744300 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.744266 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6qhi3nl02ho85\"" Jul 09 04:13:27.744382 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.744343 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 09 04:13:27.746681 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.746660 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 09 04:13:27.748467 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.748450 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 09 04:13:27.756534 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.756516 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 04:13:27.899704 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.899677 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1252f86-581d-4394-a3ac-d34d38b772e0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:27.900041 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.899710 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:27.900041 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.899731 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:27.900041 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.899749 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c1252f86-581d-4394-a3ac-d34d38b772e0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:27.900041 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.899766 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c1252f86-581d-4394-a3ac-d34d38b772e0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:27.900041 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.899780 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1252f86-581d-4394-a3ac-d34d38b772e0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:27.900041 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.899805 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:27.900041 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.899849 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c1252f86-581d-4394-a3ac-d34d38b772e0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:27.900041 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.899878 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:27.900041 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.899910 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c1252f86-581d-4394-a3ac-d34d38b772e0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:27.900041 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.899935 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:27.900041 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.899953 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-config\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:27.900041 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.899967 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-web-config\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:27.900041 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.900004 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1252f86-581d-4394-a3ac-d34d38b772e0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:27.900041 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.900021 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:27.900041 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.900045 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxkvf\" (UniqueName: \"kubernetes.io/projected/c1252f86-581d-4394-a3ac-d34d38b772e0-kube-api-access-kxkvf\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:27.900551 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.900068 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:27.900551 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.900092 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c1252f86-581d-4394-a3ac-d34d38b772e0-config-out\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:27.900551 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.900113 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c1252f86-581d-4394-a3ac-d34d38b772e0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:27.900551 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:27.900137 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1252f86-581d-4394-a3ac-d34d38b772e0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.001070 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.000993 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.001070 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.001043 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c1252f86-581d-4394-a3ac-d34d38b772e0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.001070 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.001071 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.001313 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.001092 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c1252f86-581d-4394-a3ac-d34d38b772e0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.001313 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.001113 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.001313 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.001134 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-config\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.001313 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.001156 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-web-config\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.001506 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.001465 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1252f86-581d-4394-a3ac-d34d38b772e0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.001558 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.001501 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.001558 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.001552 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxkvf\" (UniqueName: \"kubernetes.io/projected/c1252f86-581d-4394-a3ac-d34d38b772e0-kube-api-access-kxkvf\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.001659 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.001582 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.001659 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.001612 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c1252f86-581d-4394-a3ac-d34d38b772e0-config-out\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.001659 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.001641 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c1252f86-581d-4394-a3ac-d34d38b772e0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.001810 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.001783 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c1252f86-581d-4394-a3ac-d34d38b772e0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.002456 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.002429 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1252f86-581d-4394-a3ac-d34d38b772e0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.002713 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.001645 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c1252f86-581d-4394-a3ac-d34d38b772e0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.002713 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.002503 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1252f86-581d-4394-a3ac-d34d38b772e0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.002713 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.002540 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1252f86-581d-4394-a3ac-d34d38b772e0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.002713 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.002564 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.002713 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.002593 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.002713 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.002626 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c1252f86-581d-4394-a3ac-d34d38b772e0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.002713 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.002651 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c1252f86-581d-4394-a3ac-d34d38b772e0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.002713 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.002673 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1252f86-581d-4394-a3ac-d34d38b772e0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.004036 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.003751 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1252f86-581d-4394-a3ac-d34d38b772e0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.004246 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.004210 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-web-config\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.004358 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.004335 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.005651 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.004557 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.005651 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.004704 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.005651 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.004932 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-config\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.005651 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.005138 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1252f86-581d-4394-a3ac-d34d38b772e0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.005651 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.005601 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c1252f86-581d-4394-a3ac-d34d38b772e0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.005944 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.005842 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.005998 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.005945 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1252f86-581d-4394-a3ac-d34d38b772e0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.006339 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.006322 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c1252f86-581d-4394-a3ac-d34d38b772e0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.006800 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.006773 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.006990 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.006975 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.007506 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.007490 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c1252f86-581d-4394-a3ac-d34d38b772e0-config-out\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.007726 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.007706 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c1252f86-581d-4394-a3ac-d34d38b772e0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.008461 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.008441 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c1252f86-581d-4394-a3ac-d34d38b772e0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.012199 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.012176 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxkvf\" (UniqueName: \"kubernetes.io/projected/c1252f86-581d-4394-a3ac-d34d38b772e0-kube-api-access-kxkvf\") pod \"prometheus-k8s-0\" (UID: \"c1252f86-581d-4394-a3ac-d34d38b772e0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.049720 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.049693 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:28.190546 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.190510 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 04:13:28.194222 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:13:28.194193 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1252f86_581d_4394_a3ac_d34d38b772e0.slice/crio-140843b4dd5161134fb1666b47ec5aee0f4e74345f85d0aa8445da79f788aa20 WatchSource:0}: Error finding container 140843b4dd5161134fb1666b47ec5aee0f4e74345f85d0aa8445da79f788aa20: Status 404 returned error can't find the container with id 140843b4dd5161134fb1666b47ec5aee0f4e74345f85d0aa8445da79f788aa20 Jul 09 04:13:28.250000 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.249979 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2433376d-b2a6-424d-a06a-bf39dbb88b72" path="/var/lib/kubelet/pods/2433376d-b2a6-424d-a06a-bf39dbb88b72/volumes" Jul 09 04:13:28.670207 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.670168 2534 generic.go:356] "Generic (PLEG): container finished" podID="c1252f86-581d-4394-a3ac-d34d38b772e0" containerID="7d2bad931949720aaf591592b92765ba5f78cfbde0588c53f9584f8deef02a4c" exitCode=0 Jul 09 04:13:28.670207 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.670205 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c1252f86-581d-4394-a3ac-d34d38b772e0","Type":"ContainerDied","Data":"7d2bad931949720aaf591592b92765ba5f78cfbde0588c53f9584f8deef02a4c"} Jul 09 04:13:28.670478 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:28.670263 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c1252f86-581d-4394-a3ac-d34d38b772e0","Type":"ContainerStarted","Data":"140843b4dd5161134fb1666b47ec5aee0f4e74345f85d0aa8445da79f788aa20"} Jul 09 04:13:29.684858 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:29.684824 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c1252f86-581d-4394-a3ac-d34d38b772e0","Type":"ContainerStarted","Data":"2ffc1964ec9dd87be61de557f659bb491d91bd830ce2d7332984d24881c21691"} Jul 09 04:13:29.685190 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:29.684915 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c1252f86-581d-4394-a3ac-d34d38b772e0","Type":"ContainerStarted","Data":"e20d77d69d68e19f2689dd2b8498382037884bff3ae5bb5f98722d3e93f84955"} Jul 09 04:13:29.685190 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:29.684943 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c1252f86-581d-4394-a3ac-d34d38b772e0","Type":"ContainerStarted","Data":"e44722260855ab2ebf8d0f132e20bb574f09e0c70bd1ec53c484c4045575481c"} Jul 09 04:13:29.685190 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:29.684970 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c1252f86-581d-4394-a3ac-d34d38b772e0","Type":"ContainerStarted","Data":"2ec1415a711496b88d177a0c13f4675779a7603392dc69d5705bb0c33f2f1bd2"} Jul 09 04:13:29.685190 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:29.684997 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c1252f86-581d-4394-a3ac-d34d38b772e0","Type":"ContainerStarted","Data":"7590d747bf62e778fa7a9a336c7eb56b7dd49872817de8cd32f6df7bdfe8eb5b"} Jul 09 04:13:29.685190 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:29.685025 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c1252f86-581d-4394-a3ac-d34d38b772e0","Type":"ContainerStarted","Data":"5d42af3be4c67ed4bc273f020a5678fffb6b1ab3c613ed87a795b0b8f2e7383d"} Jul 09 04:13:29.715504 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:29.715452 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.694783207 podStartE2EDuration="2.715432709s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.649502ms" isStatefulPod=false podCreationTimestamp="2026-07-09 04:13:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:13:29.713708558 +0000 UTC m=+399.937327057" watchObservedRunningTime="2026-07-09 04:13:29.715432709 +0000 UTC m=+399.939051197" Jul 09 04:13:33.050362 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:33.050331 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:13:35.091618 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:13:35.091574 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" podUID="9d2284ee-ac3c-4e3c-9b08-67abc9f14e75" Jul 09 04:13:35.710221 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:35.710191 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" Jul 09 04:13:37.688785 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:37.688749 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-z5c2w\" (UID: \"9d2284ee-ac3c-4e3c-9b08-67abc9f14e75\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" Jul 09 04:13:37.691226 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:37.691205 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9d2284ee-ac3c-4e3c-9b08-67abc9f14e75-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-z5c2w\" (UID: \"9d2284ee-ac3c-4e3c-9b08-67abc9f14e75\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" Jul 09 04:13:37.814156 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:37.814131 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-59nnd\"" Jul 09 04:13:37.821149 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:37.821128 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" Jul 09 04:13:37.949669 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:37.949637 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w"] Jul 09 04:13:37.952390 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:13:37.952359 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d2284ee_ac3c_4e3c_9b08_67abc9f14e75.slice/crio-e77d405ca516138174da1d6b0f598f0dfd2011559e23d96a3b208db3c7cbd6ec WatchSource:0}: Error finding container e77d405ca516138174da1d6b0f598f0dfd2011559e23d96a3b208db3c7cbd6ec: Status 404 returned error can't find the container with id e77d405ca516138174da1d6b0f598f0dfd2011559e23d96a3b208db3c7cbd6ec Jul 09 04:13:38.727906 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:38.726718 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" event={"ID":"9d2284ee-ac3c-4e3c-9b08-67abc9f14e75","Type":"ContainerStarted","Data":"e77d405ca516138174da1d6b0f598f0dfd2011559e23d96a3b208db3c7cbd6ec"} Jul 09 04:13:39.732048 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:39.732019 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" event={"ID":"9d2284ee-ac3c-4e3c-9b08-67abc9f14e75","Type":"ContainerStarted","Data":"1ba97cf28e00acc43f301eecc90b57e00a9946abdac8c9953b5199b3d2efde34"} Jul 09 04:13:39.747643 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:13:39.747589 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-z5c2w" podStartSLOduration=400.652444608 podStartE2EDuration="6m41.747574047s" totalImagesPullingTime="1.095129439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:06:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:13:39.745812038 +0000 UTC m=+409.969430526" watchObservedRunningTime="2026-07-09 04:13:39.747574047 +0000 UTC m=+409.971192535" Jul 09 04:14:28.050234 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:14:28.050197 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:14:28.066660 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:14:28.066632 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:14:28.947307 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:14:28.947265 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:15:37.756693 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:37.756660 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg"] Jul 09 04:15:37.760107 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:37.760081 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg" Jul 09 04:15:37.763200 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:37.763175 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4xqf2\"" Jul 09 04:15:37.763356 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:37.763317 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 04:15:37.763700 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:37.763682 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 04:15:37.763798 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:37.763683 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 04:15:37.764202 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:37.764182 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 04:15:37.781839 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:37.781807 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg"] Jul 09 04:15:37.810042 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:37.810013 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a2d3035-3a6a-4db4-bc3f-de7a6c360ded-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cff547d94-vs2gg\" (UID: \"5a2d3035-3a6a-4db4-bc3f-de7a6c360ded\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg" Jul 09 04:15:37.810200 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:37.810064 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5a2d3035-3a6a-4db4-bc3f-de7a6c360ded-manifests\") pod \"opendatahub-operator-controller-manager-5cff547d94-vs2gg\" (UID: \"5a2d3035-3a6a-4db4-bc3f-de7a6c360ded\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg" Jul 09 04:15:37.810200 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:37.810086 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5a2d3035-3a6a-4db4-bc3f-de7a6c360ded-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cff547d94-vs2gg\" (UID: \"5a2d3035-3a6a-4db4-bc3f-de7a6c360ded\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg" Jul 09 04:15:37.810200 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:37.810173 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc6hn\" (UniqueName: \"kubernetes.io/projected/5a2d3035-3a6a-4db4-bc3f-de7a6c360ded-kube-api-access-jc6hn\") pod \"opendatahub-operator-controller-manager-5cff547d94-vs2gg\" (UID: \"5a2d3035-3a6a-4db4-bc3f-de7a6c360ded\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg" Jul 09 04:15:37.810328 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:37.810211 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a2d3035-3a6a-4db4-bc3f-de7a6c360ded-tmp\") pod \"opendatahub-operator-controller-manager-5cff547d94-vs2gg\" (UID: \"5a2d3035-3a6a-4db4-bc3f-de7a6c360ded\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg" Jul 09 04:15:37.911142 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:37.911100 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5a2d3035-3a6a-4db4-bc3f-de7a6c360ded-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cff547d94-vs2gg\" (UID: \"5a2d3035-3a6a-4db4-bc3f-de7a6c360ded\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg" Jul 09 04:15:37.911363 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:37.911181 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc6hn\" (UniqueName: \"kubernetes.io/projected/5a2d3035-3a6a-4db4-bc3f-de7a6c360ded-kube-api-access-jc6hn\") pod \"opendatahub-operator-controller-manager-5cff547d94-vs2gg\" (UID: \"5a2d3035-3a6a-4db4-bc3f-de7a6c360ded\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg" Jul 09 04:15:37.911363 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:37.911222 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a2d3035-3a6a-4db4-bc3f-de7a6c360ded-tmp\") pod \"opendatahub-operator-controller-manager-5cff547d94-vs2gg\" (UID: \"5a2d3035-3a6a-4db4-bc3f-de7a6c360ded\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg" Jul 09 04:15:37.911363 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:37.911252 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a2d3035-3a6a-4db4-bc3f-de7a6c360ded-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cff547d94-vs2gg\" (UID: \"5a2d3035-3a6a-4db4-bc3f-de7a6c360ded\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg" Jul 09 04:15:37.911363 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:37.911333 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5a2d3035-3a6a-4db4-bc3f-de7a6c360ded-manifests\") pod \"opendatahub-operator-controller-manager-5cff547d94-vs2gg\" (UID: \"5a2d3035-3a6a-4db4-bc3f-de7a6c360ded\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg" Jul 09 04:15:37.911773 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:37.911752 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5a2d3035-3a6a-4db4-bc3f-de7a6c360ded-manifests\") pod \"opendatahub-operator-controller-manager-5cff547d94-vs2gg\" (UID: \"5a2d3035-3a6a-4db4-bc3f-de7a6c360ded\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg" Jul 09 04:15:37.916821 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:37.916790 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a2d3035-3a6a-4db4-bc3f-de7a6c360ded-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cff547d94-vs2gg\" (UID: \"5a2d3035-3a6a-4db4-bc3f-de7a6c360ded\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg" Jul 09 04:15:37.919098 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:37.919063 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5a2d3035-3a6a-4db4-bc3f-de7a6c360ded-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cff547d94-vs2gg\" (UID: \"5a2d3035-3a6a-4db4-bc3f-de7a6c360ded\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg" Jul 09 04:15:37.919925 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:37.919899 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a2d3035-3a6a-4db4-bc3f-de7a6c360ded-tmp\") pod \"opendatahub-operator-controller-manager-5cff547d94-vs2gg\" (UID: \"5a2d3035-3a6a-4db4-bc3f-de7a6c360ded\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg" Jul 09 04:15:37.920854 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:37.920830 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc6hn\" (UniqueName: \"kubernetes.io/projected/5a2d3035-3a6a-4db4-bc3f-de7a6c360ded-kube-api-access-jc6hn\") pod \"opendatahub-operator-controller-manager-5cff547d94-vs2gg\" (UID: \"5a2d3035-3a6a-4db4-bc3f-de7a6c360ded\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg" Jul 09 04:15:38.072664 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:38.072633 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg" Jul 09 04:15:38.219219 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:38.219172 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg"] Jul 09 04:15:38.222463 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:15:38.222436 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a2d3035_3a6a_4db4_bc3f_de7a6c360ded.slice/crio-b51bf186925bb7ac54536e472f9e61e4505f68fd6e475b44e2b504c4660856f1 WatchSource:0}: Error finding container b51bf186925bb7ac54536e472f9e61e4505f68fd6e475b44e2b504c4660856f1: Status 404 returned error can't find the container with id b51bf186925bb7ac54536e472f9e61e4505f68fd6e475b44e2b504c4660856f1 Jul 09 04:15:39.230683 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:39.230600 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg" event={"ID":"5a2d3035-3a6a-4db4-bc3f-de7a6c360ded","Type":"ContainerStarted","Data":"b51bf186925bb7ac54536e472f9e61e4505f68fd6e475b44e2b504c4660856f1"} Jul 09 04:15:42.249507 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:42.249477 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg" event={"ID":"5a2d3035-3a6a-4db4-bc3f-de7a6c360ded","Type":"ContainerStarted","Data":"802d92d388ecddc1e45854100ac625dc89929a197f01c691aa220f643943e1e1"} Jul 09 04:15:43.253170 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:43.253128 2534 generic.go:356] "Generic (PLEG): container finished" podID="5a2d3035-3a6a-4db4-bc3f-de7a6c360ded" containerID="802d92d388ecddc1e45854100ac625dc89929a197f01c691aa220f643943e1e1" exitCode=0 Jul 09 04:15:43.253678 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:43.253219 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg" event={"ID":"5a2d3035-3a6a-4db4-bc3f-de7a6c360ded","Type":"ContainerDied","Data":"802d92d388ecddc1e45854100ac625dc89929a197f01c691aa220f643943e1e1"} Jul 09 04:15:44.261494 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:44.261461 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg" event={"ID":"5a2d3035-3a6a-4db4-bc3f-de7a6c360ded","Type":"ContainerStarted","Data":"1c13e47ccea666b11837d11cb0ead985421579af5a87464f0f0d71f4871fa711"} Jul 09 04:15:44.261979 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:44.261562 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg" Jul 09 04:15:44.284254 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:44.284201 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg" podStartSLOduration=2.750658061 podStartE2EDuration="7.284183609s" totalImagesPullingTime="3.515813236s" totalInitContainerRuntime="1.017712312s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:15:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:15:44.282371594 +0000 UTC m=+534.505990082" watchObservedRunningTime="2026-07-09 04:15:44.284183609 +0000 UTC m=+534.507802097" Jul 09 04:15:55.270852 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:15:55.270823 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-vs2gg" Jul 09 04:16:04.208893 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.208846 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj"] Jul 09 04:16:04.221257 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.221225 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.223650 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.223626 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 04:16:04.223818 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.223746 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 04:16:04.223818 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.223755 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 04:16:04.224037 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.224017 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-sfmq7\"" Jul 09 04:16:04.227107 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.227072 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj"] Jul 09 04:16:04.358747 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.358715 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.358928 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.358761 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.358928 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.358802 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.358928 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.358835 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.358928 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.358852 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.358928 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.358867 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv76j\" (UniqueName: \"kubernetes.io/projected/d20410bf-579e-4a3c-b8f0-43debad6d9ee-kube-api-access-jv76j\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.358928 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.358907 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.358928 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.358926 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.359193 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.359012 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.459892 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.459794 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.459892 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.459856 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.460124 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.459896 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.460124 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.459922 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.460124 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.459943 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.460124 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.459967 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv76j\" (UniqueName: \"kubernetes.io/projected/d20410bf-579e-4a3c-b8f0-43debad6d9ee-kube-api-access-jv76j\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.460124 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.460007 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.460124 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.460034 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.460124 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.460083 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.460502 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.460360 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.460502 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.460365 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.460614 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.460500 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.460614 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.460557 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.460751 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.460726 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.462596 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.462571 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.462732 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.462717 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.468023 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.467993 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv76j\" (UniqueName: \"kubernetes.io/projected/d20410bf-579e-4a3c-b8f0-43debad6d9ee-kube-api-access-jv76j\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.468420 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.468401 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.534814 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.534774 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:04.699589 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:04.699561 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj"] Jul 09 04:16:04.702929 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:16:04.702889 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd20410bf_579e_4a3c_b8f0_43debad6d9ee.slice/crio-92c917b6a8a6cc65f4f191163fa762be8e0feac1b396a6cdcbf695a20d1ce579 WatchSource:0}: Error finding container 92c917b6a8a6cc65f4f191163fa762be8e0feac1b396a6cdcbf695a20d1ce579: Status 404 returned error can't find the container with id 92c917b6a8a6cc65f4f191163fa762be8e0feac1b396a6cdcbf695a20d1ce579 Jul 09 04:16:05.211754 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.211720 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx"] Jul 09 04:16:05.221468 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.221440 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" Jul 09 04:16:05.224351 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.224321 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 09 04:16:05.224502 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.224387 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-4l8v4\"" Jul 09 04:16:05.224581 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.224555 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 09 04:16:05.224634 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.224564 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 09 04:16:05.226563 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.226540 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx"] Jul 09 04:16:05.362740 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.362701 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" event={"ID":"d20410bf-579e-4a3c-b8f0-43debad6d9ee","Type":"ContainerStarted","Data":"92c917b6a8a6cc65f4f191163fa762be8e0feac1b396a6cdcbf695a20d1ce579"} Jul 09 04:16:05.370161 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.370124 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/90f3ac8b-482f-439f-ae20-05262ae79c22-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6wqjx\" (UID: \"90f3ac8b-482f-439f-ae20-05262ae79c22\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" Jul 09 04:16:05.370321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.370164 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/90f3ac8b-482f-439f-ae20-05262ae79c22-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6wqjx\" (UID: \"90f3ac8b-482f-439f-ae20-05262ae79c22\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" Jul 09 04:16:05.370321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.370208 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/90f3ac8b-482f-439f-ae20-05262ae79c22-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6wqjx\" (UID: \"90f3ac8b-482f-439f-ae20-05262ae79c22\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" Jul 09 04:16:05.370321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.370234 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/90f3ac8b-482f-439f-ae20-05262ae79c22-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6wqjx\" (UID: \"90f3ac8b-482f-439f-ae20-05262ae79c22\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" Jul 09 04:16:05.370467 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.370318 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4msq\" (UniqueName: \"kubernetes.io/projected/90f3ac8b-482f-439f-ae20-05262ae79c22-kube-api-access-r4msq\") pod \"istiod-openshift-gateway-94bb8fbfd-6wqjx\" (UID: \"90f3ac8b-482f-439f-ae20-05262ae79c22\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" Jul 09 04:16:05.370467 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.370373 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90f3ac8b-482f-439f-ae20-05262ae79c22-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6wqjx\" (UID: \"90f3ac8b-482f-439f-ae20-05262ae79c22\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" Jul 09 04:16:05.370467 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.370397 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/90f3ac8b-482f-439f-ae20-05262ae79c22-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6wqjx\" (UID: \"90f3ac8b-482f-439f-ae20-05262ae79c22\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" Jul 09 04:16:05.471340 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.471203 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/90f3ac8b-482f-439f-ae20-05262ae79c22-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6wqjx\" (UID: \"90f3ac8b-482f-439f-ae20-05262ae79c22\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" Jul 09 04:16:05.471340 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.471254 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/90f3ac8b-482f-439f-ae20-05262ae79c22-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6wqjx\" (UID: \"90f3ac8b-482f-439f-ae20-05262ae79c22\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" Jul 09 04:16:05.471340 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.471324 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4msq\" (UniqueName: \"kubernetes.io/projected/90f3ac8b-482f-439f-ae20-05262ae79c22-kube-api-access-r4msq\") pod \"istiod-openshift-gateway-94bb8fbfd-6wqjx\" (UID: \"90f3ac8b-482f-439f-ae20-05262ae79c22\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" Jul 09 04:16:05.471630 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.471365 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90f3ac8b-482f-439f-ae20-05262ae79c22-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6wqjx\" (UID: \"90f3ac8b-482f-439f-ae20-05262ae79c22\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" Jul 09 04:16:05.471630 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.471399 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/90f3ac8b-482f-439f-ae20-05262ae79c22-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6wqjx\" (UID: \"90f3ac8b-482f-439f-ae20-05262ae79c22\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" Jul 09 04:16:05.471630 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.471479 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/90f3ac8b-482f-439f-ae20-05262ae79c22-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6wqjx\" (UID: \"90f3ac8b-482f-439f-ae20-05262ae79c22\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" Jul 09 04:16:05.471630 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.471502 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/90f3ac8b-482f-439f-ae20-05262ae79c22-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6wqjx\" (UID: \"90f3ac8b-482f-439f-ae20-05262ae79c22\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" Jul 09 04:16:05.472215 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.472179 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/90f3ac8b-482f-439f-ae20-05262ae79c22-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6wqjx\" (UID: \"90f3ac8b-482f-439f-ae20-05262ae79c22\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" Jul 09 04:16:05.473829 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.473804 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/90f3ac8b-482f-439f-ae20-05262ae79c22-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6wqjx\" (UID: \"90f3ac8b-482f-439f-ae20-05262ae79c22\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" Jul 09 04:16:05.474019 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.473999 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/90f3ac8b-482f-439f-ae20-05262ae79c22-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6wqjx\" (UID: \"90f3ac8b-482f-439f-ae20-05262ae79c22\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" Jul 09 04:16:05.474339 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.474319 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/90f3ac8b-482f-439f-ae20-05262ae79c22-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6wqjx\" (UID: \"90f3ac8b-482f-439f-ae20-05262ae79c22\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" Jul 09 04:16:05.474339 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.474334 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/90f3ac8b-482f-439f-ae20-05262ae79c22-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6wqjx\" (UID: \"90f3ac8b-482f-439f-ae20-05262ae79c22\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" Jul 09 04:16:05.482695 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.482672 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90f3ac8b-482f-439f-ae20-05262ae79c22-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6wqjx\" (UID: \"90f3ac8b-482f-439f-ae20-05262ae79c22\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" Jul 09 04:16:05.482800 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.482707 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4msq\" (UniqueName: \"kubernetes.io/projected/90f3ac8b-482f-439f-ae20-05262ae79c22-kube-api-access-r4msq\") pod \"istiod-openshift-gateway-94bb8fbfd-6wqjx\" (UID: \"90f3ac8b-482f-439f-ae20-05262ae79c22\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" Jul 09 04:16:05.532738 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.532700 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" Jul 09 04:16:05.728250 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:05.728225 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx"] Jul 09 04:16:05.731917 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:16:05.731611 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90f3ac8b_482f_439f_ae20_05262ae79c22.slice/crio-9d3f864eecb855824c5b0d2c5ccdc17ac41a1eb8e02e03986d97f75f3bd1852b WatchSource:0}: Error finding container 9d3f864eecb855824c5b0d2c5ccdc17ac41a1eb8e02e03986d97f75f3bd1852b: Status 404 returned error can't find the container with id 9d3f864eecb855824c5b0d2c5ccdc17ac41a1eb8e02e03986d97f75f3bd1852b Jul 09 04:16:06.372007 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:06.371964 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" event={"ID":"90f3ac8b-482f-439f-ae20-05262ae79c22","Type":"ContainerStarted","Data":"9d3f864eecb855824c5b0d2c5ccdc17ac41a1eb8e02e03986d97f75f3bd1852b"} Jul 09 04:16:07.621727 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:07.621685 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-645bc6d555-5fbz6"] Jul 09 04:16:07.625391 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:07.625367 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-645bc6d555-5fbz6" Jul 09 04:16:07.631179 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:07.631153 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 09 04:16:07.631179 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:07.631166 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 09 04:16:07.632484 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:07.632229 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bst2k\"" Jul 09 04:16:07.632484 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:07.632236 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 09 04:16:07.632484 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:07.632339 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 09 04:16:07.632484 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:07.632389 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 09 04:16:07.647905 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:07.647880 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-645bc6d555-5fbz6"] Jul 09 04:16:07.803039 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:07.803001 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svncs\" (UniqueName: \"kubernetes.io/projected/88bc6fa0-72c6-4142-a63c-22eae1310672-kube-api-access-svncs\") pod \"lws-controller-manager-645bc6d555-5fbz6\" (UID: \"88bc6fa0-72c6-4142-a63c-22eae1310672\") " pod="openshift-lws-operator/lws-controller-manager-645bc6d555-5fbz6" Jul 09 04:16:07.803039 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:07.803044 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/88bc6fa0-72c6-4142-a63c-22eae1310672-cert\") pod \"lws-controller-manager-645bc6d555-5fbz6\" (UID: \"88bc6fa0-72c6-4142-a63c-22eae1310672\") " pod="openshift-lws-operator/lws-controller-manager-645bc6d555-5fbz6" Jul 09 04:16:07.803277 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:07.803152 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/88bc6fa0-72c6-4142-a63c-22eae1310672-metrics-cert\") pod \"lws-controller-manager-645bc6d555-5fbz6\" (UID: \"88bc6fa0-72c6-4142-a63c-22eae1310672\") " pod="openshift-lws-operator/lws-controller-manager-645bc6d555-5fbz6" Jul 09 04:16:07.803277 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:07.803206 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/88bc6fa0-72c6-4142-a63c-22eae1310672-manager-config\") pod \"lws-controller-manager-645bc6d555-5fbz6\" (UID: \"88bc6fa0-72c6-4142-a63c-22eae1310672\") " pod="openshift-lws-operator/lws-controller-manager-645bc6d555-5fbz6" Jul 09 04:16:07.903847 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:07.903732 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/88bc6fa0-72c6-4142-a63c-22eae1310672-manager-config\") pod \"lws-controller-manager-645bc6d555-5fbz6\" (UID: \"88bc6fa0-72c6-4142-a63c-22eae1310672\") " pod="openshift-lws-operator/lws-controller-manager-645bc6d555-5fbz6" Jul 09 04:16:07.904080 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:07.903845 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svncs\" (UniqueName: \"kubernetes.io/projected/88bc6fa0-72c6-4142-a63c-22eae1310672-kube-api-access-svncs\") pod \"lws-controller-manager-645bc6d555-5fbz6\" (UID: \"88bc6fa0-72c6-4142-a63c-22eae1310672\") " pod="openshift-lws-operator/lws-controller-manager-645bc6d555-5fbz6" Jul 09 04:16:07.904080 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:07.903876 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/88bc6fa0-72c6-4142-a63c-22eae1310672-cert\") pod \"lws-controller-manager-645bc6d555-5fbz6\" (UID: \"88bc6fa0-72c6-4142-a63c-22eae1310672\") " pod="openshift-lws-operator/lws-controller-manager-645bc6d555-5fbz6" Jul 09 04:16:07.904080 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:07.903940 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/88bc6fa0-72c6-4142-a63c-22eae1310672-metrics-cert\") pod \"lws-controller-manager-645bc6d555-5fbz6\" (UID: \"88bc6fa0-72c6-4142-a63c-22eae1310672\") " pod="openshift-lws-operator/lws-controller-manager-645bc6d555-5fbz6" Jul 09 04:16:07.905629 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:07.905600 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/88bc6fa0-72c6-4142-a63c-22eae1310672-manager-config\") pod \"lws-controller-manager-645bc6d555-5fbz6\" (UID: \"88bc6fa0-72c6-4142-a63c-22eae1310672\") " pod="openshift-lws-operator/lws-controller-manager-645bc6d555-5fbz6" Jul 09 04:16:07.912151 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:07.912119 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/88bc6fa0-72c6-4142-a63c-22eae1310672-cert\") pod \"lws-controller-manager-645bc6d555-5fbz6\" (UID: \"88bc6fa0-72c6-4142-a63c-22eae1310672\") " pod="openshift-lws-operator/lws-controller-manager-645bc6d555-5fbz6" Jul 09 04:16:07.927315 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:07.922019 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/88bc6fa0-72c6-4142-a63c-22eae1310672-metrics-cert\") pod \"lws-controller-manager-645bc6d555-5fbz6\" (UID: \"88bc6fa0-72c6-4142-a63c-22eae1310672\") " pod="openshift-lws-operator/lws-controller-manager-645bc6d555-5fbz6" Jul 09 04:16:07.927315 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:07.926857 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svncs\" (UniqueName: \"kubernetes.io/projected/88bc6fa0-72c6-4142-a63c-22eae1310672-kube-api-access-svncs\") pod \"lws-controller-manager-645bc6d555-5fbz6\" (UID: \"88bc6fa0-72c6-4142-a63c-22eae1310672\") " pod="openshift-lws-operator/lws-controller-manager-645bc6d555-5fbz6" Jul 09 04:16:07.944240 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:07.937474 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-645bc6d555-5fbz6" Jul 09 04:16:08.970185 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:08.970145 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:16:08.970607 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:08.970235 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:16:08.970607 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:08.970268 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:16:08.976655 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:08.976622 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:16:08.976782 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:08.976695 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:16:09.117049 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:09.116755 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-645bc6d555-5fbz6"] Jul 09 04:16:09.120414 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:16:09.120376 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88bc6fa0_72c6_4142_a63c_22eae1310672.slice/crio-8aa87ac80079c18b54d2fafc32b9b9bb8039ef4f742ab86c4ce6057059ce9b2c WatchSource:0}: Error finding container 8aa87ac80079c18b54d2fafc32b9b9bb8039ef4f742ab86c4ce6057059ce9b2c: Status 404 returned error can't find the container with id 8aa87ac80079c18b54d2fafc32b9b9bb8039ef4f742ab86c4ce6057059ce9b2c Jul 09 04:16:09.393510 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:09.393465 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" event={"ID":"d20410bf-579e-4a3c-b8f0-43debad6d9ee","Type":"ContainerStarted","Data":"92e050b478f238464450d11527ed583d994f4e5e924f1a98bd69ae1087644960"} Jul 09 04:16:09.395380 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:09.395342 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" event={"ID":"90f3ac8b-482f-439f-ae20-05262ae79c22","Type":"ContainerStarted","Data":"ee1e1f2ab403c55aeecafeff0326f2ed23ac67e4dcd76eb4dbd4c793ea6a298b"} Jul 09 04:16:09.395514 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:09.395404 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" Jul 09 04:16:09.396773 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:09.396752 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-645bc6d555-5fbz6" event={"ID":"88bc6fa0-72c6-4142-a63c-22eae1310672","Type":"ContainerStarted","Data":"8aa87ac80079c18b54d2fafc32b9b9bb8039ef4f742ab86c4ce6057059ce9b2c"} Jul 09 04:16:09.449140 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:09.449065 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" podStartSLOduration=1.184167311 podStartE2EDuration="5.449042148s" totalImagesPullingTime="4.264874837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:16:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:16:09.42067226 +0000 UTC m=+559.644290749" watchObservedRunningTime="2026-07-09 04:16:09.449042148 +0000 UTC m=+559.672660636" Jul 09 04:16:09.450121 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:09.450079 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" podStartSLOduration=1.208101699 podStartE2EDuration="4.450066296s" totalImagesPullingTime="3.241964597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:16:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:16:09.44780977 +0000 UTC m=+559.671428259" watchObservedRunningTime="2026-07-09 04:16:09.450066296 +0000 UTC m=+559.673684783" Jul 09 04:16:09.535833 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:09.535798 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:10.266471 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:10.266437 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj"] Jul 09 04:16:10.403747 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:10.403721 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wqjx" Jul 09 04:16:10.537344 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:10.537317 2534 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" start-of-body= Jul 09 04:16:10.537507 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:10.537376 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" podUID="d20410bf-579e-4a3c-b8f0-43debad6d9ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" Jul 09 04:16:11.407857 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:11.407793 2534 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" podUID="d20410bf-579e-4a3c-b8f0-43debad6d9ee" containerName="istio-proxy" containerID="cri-o://92e050b478f238464450d11527ed583d994f4e5e924f1a98bd69ae1087644960" gracePeriod=30 Jul 09 04:16:12.415151 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:12.415109 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-645bc6d555-5fbz6" event={"ID":"88bc6fa0-72c6-4142-a63c-22eae1310672","Type":"ContainerStarted","Data":"90ec5cfd08d91bac538b7517a29a2767a019c8f2a54a00b156ae09503f3b470e"} Jul 09 04:16:12.415634 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:12.415180 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-645bc6d555-5fbz6" Jul 09 04:16:12.435184 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:12.435122 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-645bc6d555-5fbz6" podStartSLOduration=3.050628986 podStartE2EDuration="5.435102212s" totalImagesPullingTime="2.384473226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:16:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:16:12.43299063 +0000 UTC m=+562.656609118" watchObservedRunningTime="2026-07-09 04:16:12.435102212 +0000 UTC m=+562.658720702" Jul 09 04:16:16.662750 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.662724 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:16.676265 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.676193 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istiod-ca-cert\") pod \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " Jul 09 04:16:16.676265 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.676239 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istio-data\") pod \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " Jul 09 04:16:16.676480 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.676294 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istio-token\") pod \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " Jul 09 04:16:16.676556 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.676531 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istio-podinfo\") pod \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " Jul 09 04:16:16.676613 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.676581 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istio-envoy\") pod \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " Jul 09 04:16:16.676613 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.676601 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d20410bf-579e-4a3c-b8f0-43debad6d9ee" (UID: "d20410bf-579e-4a3c-b8f0-43debad6d9ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:16:16.676718 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.676635 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-credential-socket\") pod \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " Jul 09 04:16:16.676718 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.676662 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-workload-certs\") pod \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " Jul 09 04:16:16.676718 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.676704 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-workload-socket\") pod \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " Jul 09 04:16:16.676861 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.676752 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jv76j\" (UniqueName: \"kubernetes.io/projected/d20410bf-579e-4a3c-b8f0-43debad6d9ee-kube-api-access-jv76j\") pod \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\" (UID: \"d20410bf-579e-4a3c-b8f0-43debad6d9ee\") " Jul 09 04:16:16.677075 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.677056 2534 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istiod-ca-cert\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:16:16.677137 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.677076 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d20410bf-579e-4a3c-b8f0-43debad6d9ee" (UID: "d20410bf-579e-4a3c-b8f0-43debad6d9ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:16:16.677578 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.677543 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d20410bf-579e-4a3c-b8f0-43debad6d9ee" (UID: "d20410bf-579e-4a3c-b8f0-43debad6d9ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:16:16.677897 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.677808 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d20410bf-579e-4a3c-b8f0-43debad6d9ee" (UID: "d20410bf-579e-4a3c-b8f0-43debad6d9ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:16:16.678089 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.678055 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istio-data" (OuterVolumeSpecName: "istio-data") pod "d20410bf-579e-4a3c-b8f0-43debad6d9ee" (UID: "d20410bf-579e-4a3c-b8f0-43debad6d9ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:16:16.679469 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.679441 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d20410bf-579e-4a3c-b8f0-43debad6d9ee" (UID: "d20410bf-579e-4a3c-b8f0-43debad6d9ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 04:16:16.679634 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.679605 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d20410bf-579e-4a3c-b8f0-43debad6d9ee-kube-api-access-jv76j" (OuterVolumeSpecName: "kube-api-access-jv76j") pod "d20410bf-579e-4a3c-b8f0-43debad6d9ee" (UID: "d20410bf-579e-4a3c-b8f0-43debad6d9ee"). InnerVolumeSpecName "kube-api-access-jv76j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:16:16.679858 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.679838 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istio-token" (OuterVolumeSpecName: "istio-token") pod "d20410bf-579e-4a3c-b8f0-43debad6d9ee" (UID: "d20410bf-579e-4a3c-b8f0-43debad6d9ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:16:16.679941 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.679859 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d20410bf-579e-4a3c-b8f0-43debad6d9ee" (UID: "d20410bf-579e-4a3c-b8f0-43debad6d9ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:16:16.778443 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.778406 2534 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istio-podinfo\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:16:16.778443 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.778435 2534 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istio-envoy\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:16:16.778443 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.778446 2534 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-credential-socket\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:16:16.778674 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.778455 2534 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-workload-certs\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:16:16.778674 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.778464 2534 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-workload-socket\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:16:16.778674 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.778472 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jv76j\" (UniqueName: \"kubernetes.io/projected/d20410bf-579e-4a3c-b8f0-43debad6d9ee-kube-api-access-jv76j\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:16:16.778674 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.778482 2534 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istio-data\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:16:16.778674 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:16.778489 2534 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d20410bf-579e-4a3c-b8f0-43debad6d9ee-istio-token\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:16:17.440910 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:17.440876 2534 generic.go:356] "Generic (PLEG): container finished" podID="d20410bf-579e-4a3c-b8f0-43debad6d9ee" containerID="92e050b478f238464450d11527ed583d994f4e5e924f1a98bd69ae1087644960" exitCode=0 Jul 09 04:16:17.441120 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:17.440924 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" Jul 09 04:16:17.441120 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:17.440970 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" event={"ID":"d20410bf-579e-4a3c-b8f0-43debad6d9ee","Type":"ContainerDied","Data":"92e050b478f238464450d11527ed583d994f4e5e924f1a98bd69ae1087644960"} Jul 09 04:16:17.441120 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:17.441030 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj" event={"ID":"d20410bf-579e-4a3c-b8f0-43debad6d9ee","Type":"ContainerDied","Data":"92c917b6a8a6cc65f4f191163fa762be8e0feac1b396a6cdcbf695a20d1ce579"} Jul 09 04:16:17.441120 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:17.441057 2534 scope.go:117] "RemoveContainer" containerID="92e050b478f238464450d11527ed583d994f4e5e924f1a98bd69ae1087644960" Jul 09 04:16:17.451877 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:17.451858 2534 scope.go:117] "RemoveContainer" containerID="92e050b478f238464450d11527ed583d994f4e5e924f1a98bd69ae1087644960" Jul 09 04:16:17.452216 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:16:17.452192 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92e050b478f238464450d11527ed583d994f4e5e924f1a98bd69ae1087644960\": container with ID starting with 92e050b478f238464450d11527ed583d994f4e5e924f1a98bd69ae1087644960 not found: ID does not exist" containerID="92e050b478f238464450d11527ed583d994f4e5e924f1a98bd69ae1087644960" Jul 09 04:16:17.452322 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:17.452228 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92e050b478f238464450d11527ed583d994f4e5e924f1a98bd69ae1087644960"} err="failed to get container status \"92e050b478f238464450d11527ed583d994f4e5e924f1a98bd69ae1087644960\": rpc error: code = NotFound desc = could not find container \"92e050b478f238464450d11527ed583d994f4e5e924f1a98bd69ae1087644960\": container with ID starting with 92e050b478f238464450d11527ed583d994f4e5e924f1a98bd69ae1087644960 not found: ID does not exist" Jul 09 04:16:17.462315 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:17.462269 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj"] Jul 09 04:16:17.466085 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:17.466064 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-hr4kj"] Jul 09 04:16:18.250510 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:18.250472 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d20410bf-579e-4a3c-b8f0-43debad6d9ee" path="/var/lib/kubelet/pods/d20410bf-579e-4a3c-b8f0-43debad6d9ee/volumes" Jul 09 04:16:23.422183 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:23.422157 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-645bc6d555-5fbz6" Jul 09 04:16:50.189509 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:50.189483 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8dnq_633a3ae5-3d78-4217-901a-6d0d06442cf4/console-operator/1.log" Jul 09 04:16:50.192004 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:16:50.191976 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8dnq_633a3ae5-3d78-4217-901a-6d0d06442cf4/console-operator/1.log" Jul 09 04:18:01.007205 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.007173 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wgrnt"] Jul 09 04:18:01.007697 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.007559 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d20410bf-579e-4a3c-b8f0-43debad6d9ee" containerName="istio-proxy" Jul 09 04:18:01.007697 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.007586 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="d20410bf-579e-4a3c-b8f0-43debad6d9ee" containerName="istio-proxy" Jul 09 04:18:01.007697 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.007658 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="d20410bf-579e-4a3c-b8f0-43debad6d9ee" containerName="istio-proxy" Jul 09 04:18:01.009584 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.009568 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wgrnt" Jul 09 04:18:01.011892 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.011868 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 04:18:01.011892 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.011882 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-lw2qk\"" Jul 09 04:18:01.012054 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.011893 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 04:18:01.021944 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.021924 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wgrnt"] Jul 09 04:18:01.105093 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.105068 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvh24\" (UniqueName: \"kubernetes.io/projected/95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d-kube-api-access-nvh24\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wgrnt\" (UID: \"95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wgrnt" Jul 09 04:18:01.105241 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.105120 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wgrnt\" (UID: \"95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wgrnt" Jul 09 04:18:01.206050 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.206011 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wgrnt\" (UID: \"95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wgrnt" Jul 09 04:18:01.206238 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.206103 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvh24\" (UniqueName: \"kubernetes.io/projected/95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d-kube-api-access-nvh24\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wgrnt\" (UID: \"95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wgrnt" Jul 09 04:18:01.206417 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.206395 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wgrnt\" (UID: \"95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wgrnt" Jul 09 04:18:01.217523 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.217501 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvh24\" (UniqueName: \"kubernetes.io/projected/95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d-kube-api-access-nvh24\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wgrnt\" (UID: \"95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wgrnt" Jul 09 04:18:01.319435 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.319392 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wgrnt" Jul 09 04:18:01.454266 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.454231 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wgrnt"] Jul 09 04:18:01.457914 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:18:01.457877 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95cfd0ee_6cb3_450d_b3d6_58bdd8388a5d.slice/crio-327f8f5216ae82c5ab5645d48ae9cd474c9569bd3e1607ed58fc3301f133f204 WatchSource:0}: Error finding container 327f8f5216ae82c5ab5645d48ae9cd474c9569bd3e1607ed58fc3301f133f204: Status 404 returned error can't find the container with id 327f8f5216ae82c5ab5645d48ae9cd474c9569bd3e1607ed58fc3301f133f204 Jul 09 04:18:01.460165 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.460149 2534 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 04:18:01.666766 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.666686 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wgrnt"] Jul 09 04:18:01.670862 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.670832 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wgrnt"] Jul 09 04:18:01.693885 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.693860 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9v5n7"] Jul 09 04:18:01.697128 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.697109 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9v5n7" Jul 09 04:18:01.714747 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.714721 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9v5n7"] Jul 09 04:18:01.811850 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.811816 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrvcw\" (UniqueName: \"kubernetes.io/projected/f0d73dfb-3a62-475d-ae3b-01ba4799d52a-kube-api-access-wrvcw\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9v5n7\" (UID: \"f0d73dfb-3a62-475d-ae3b-01ba4799d52a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9v5n7" Jul 09 04:18:01.811999 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.811866 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0d73dfb-3a62-475d-ae3b-01ba4799d52a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9v5n7\" (UID: \"f0d73dfb-3a62-475d-ae3b-01ba4799d52a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9v5n7" Jul 09 04:18:01.912628 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.912595 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0d73dfb-3a62-475d-ae3b-01ba4799d52a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9v5n7\" (UID: \"f0d73dfb-3a62-475d-ae3b-01ba4799d52a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9v5n7" Jul 09 04:18:01.912844 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.912666 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrvcw\" (UniqueName: \"kubernetes.io/projected/f0d73dfb-3a62-475d-ae3b-01ba4799d52a-kube-api-access-wrvcw\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9v5n7\" (UID: \"f0d73dfb-3a62-475d-ae3b-01ba4799d52a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9v5n7" Jul 09 04:18:01.912978 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.912960 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0d73dfb-3a62-475d-ae3b-01ba4799d52a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9v5n7\" (UID: \"f0d73dfb-3a62-475d-ae3b-01ba4799d52a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9v5n7" Jul 09 04:18:01.927137 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:01.927069 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrvcw\" (UniqueName: \"kubernetes.io/projected/f0d73dfb-3a62-475d-ae3b-01ba4799d52a-kube-api-access-wrvcw\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9v5n7\" (UID: \"f0d73dfb-3a62-475d-ae3b-01ba4799d52a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9v5n7" Jul 09 04:18:02.011928 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:02.011879 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9v5n7" Jul 09 04:18:02.321447 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:02.321404 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9v5n7"] Jul 09 04:18:02.357799 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:18:02.357768 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0d73dfb_3a62_475d_ae3b_01ba4799d52a.slice/crio-1553b03d30c98d9d2ebd7567eca21a0d5b63b42ba268f630be9e92d513e983a9 WatchSource:0}: Error finding container 1553b03d30c98d9d2ebd7567eca21a0d5b63b42ba268f630be9e92d513e983a9: Status 404 returned error can't find the container with id 1553b03d30c98d9d2ebd7567eca21a0d5b63b42ba268f630be9e92d513e983a9 Jul 09 04:18:02.948935 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:02.948890 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9v5n7" event={"ID":"f0d73dfb-3a62-475d-ae3b-01ba4799d52a","Type":"ContainerStarted","Data":"1553b03d30c98d9d2ebd7567eca21a0d5b63b42ba268f630be9e92d513e983a9"} Jul 09 04:18:07.977591 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:07.977527 2534 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wgrnt" podUID="95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d" containerName="manager" containerID="cri-o://0ef86806a5648a9d3e9c24297ce16dcdfe5c48a0f3aeea8d49a186916e363358" gracePeriod=2 Jul 09 04:18:07.979161 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:07.979123 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9v5n7" event={"ID":"f0d73dfb-3a62-475d-ae3b-01ba4799d52a","Type":"ContainerStarted","Data":"1804a3f640edecf2e8bfd17e45e7939ea896daa2e9fef97926de3f0ef2fbc41a"} Jul 09 04:18:07.979324 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:07.979187 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9v5n7" Jul 09 04:18:08.000166 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:08.000121 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9v5n7" podStartSLOduration=2.23080469 podStartE2EDuration="7.000104414s" totalImagesPullingTime="4.769299724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:18:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:18:07.997761217 +0000 UTC m=+678.221379705" watchObservedRunningTime="2026-07-09 04:18:08.000104414 +0000 UTC m=+678.223722914" Jul 09 04:18:08.221267 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:08.221239 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wgrnt" Jul 09 04:18:08.223465 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:18:08.223439 2534 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-wgrnt\" is forbidden: User \"system:node:ip-10-0-130-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-153.ec2.internal' and this object" podUID="95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wgrnt" Jul 09 04:18:08.281138 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:08.281074 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvh24\" (UniqueName: \"kubernetes.io/projected/95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d-kube-api-access-nvh24\") pod \"95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d\" (UID: \"95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d\") " Jul 09 04:18:08.281138 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:08.281119 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d-extensions-socket-volume\") pod \"95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d\" (UID: \"95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d\") " Jul 09 04:18:08.281450 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:08.281423 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d" (UID: "95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:18:08.283562 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:08.283539 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d-kube-api-access-nvh24" (OuterVolumeSpecName: "kube-api-access-nvh24") pod "95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d" (UID: "95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d"). InnerVolumeSpecName "kube-api-access-nvh24". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:18:08.382319 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:08.382258 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvh24\" (UniqueName: \"kubernetes.io/projected/95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d-kube-api-access-nvh24\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:18:08.382319 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:08.382317 2534 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d-extensions-socket-volume\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:18:08.990738 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:08.990704 2534 generic.go:356] "Generic (PLEG): container finished" podID="95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d" containerID="0ef86806a5648a9d3e9c24297ce16dcdfe5c48a0f3aeea8d49a186916e363358" exitCode=2 Jul 09 04:18:08.991210 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:08.990749 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wgrnt" Jul 09 04:18:08.991210 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:08.990789 2534 scope.go:117] "RemoveContainer" containerID="0ef86806a5648a9d3e9c24297ce16dcdfe5c48a0f3aeea8d49a186916e363358" Jul 09 04:18:09.000344 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:09.000328 2534 scope.go:117] "RemoveContainer" containerID="0ef86806a5648a9d3e9c24297ce16dcdfe5c48a0f3aeea8d49a186916e363358" Jul 09 04:18:09.000608 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:18:09.000590 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ef86806a5648a9d3e9c24297ce16dcdfe5c48a0f3aeea8d49a186916e363358\": container with ID starting with 0ef86806a5648a9d3e9c24297ce16dcdfe5c48a0f3aeea8d49a186916e363358 not found: ID does not exist" containerID="0ef86806a5648a9d3e9c24297ce16dcdfe5c48a0f3aeea8d49a186916e363358" Jul 09 04:18:09.000663 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:09.000617 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ef86806a5648a9d3e9c24297ce16dcdfe5c48a0f3aeea8d49a186916e363358"} err="failed to get container status \"0ef86806a5648a9d3e9c24297ce16dcdfe5c48a0f3aeea8d49a186916e363358\": rpc error: code = NotFound desc = could not find container \"0ef86806a5648a9d3e9c24297ce16dcdfe5c48a0f3aeea8d49a186916e363358\": container with ID starting with 0ef86806a5648a9d3e9c24297ce16dcdfe5c48a0f3aeea8d49a186916e363358 not found: ID does not exist" Jul 09 04:18:09.001398 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:18:09.001376 2534 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-wgrnt\" is forbidden: User \"system:node:ip-10-0-130-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-153.ec2.internal' and this object" podUID="95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wgrnt" Jul 09 04:18:10.248887 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:18:10.248842 2534 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-wgrnt\" is forbidden: User \"system:node:ip-10-0-130-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-153.ec2.internal' and this object" podUID="95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wgrnt" Jul 09 04:18:10.250538 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:10.250518 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d" path="/var/lib/kubelet/pods/95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d/volumes" Jul 09 04:18:18.993465 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:18.993434 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9v5n7" Jul 09 04:18:31.330256 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:31.330219 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9v5n7"] Jul 09 04:18:31.330720 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:31.330454 2534 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9v5n7" podUID="f0d73dfb-3a62-475d-ae3b-01ba4799d52a" containerName="manager" containerID="cri-o://1804a3f640edecf2e8bfd17e45e7939ea896daa2e9fef97926de3f0ef2fbc41a" gracePeriod=10 Jul 09 04:18:31.584730 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:31.584667 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9v5n7" Jul 09 04:18:31.686650 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:31.686623 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrvcw\" (UniqueName: \"kubernetes.io/projected/f0d73dfb-3a62-475d-ae3b-01ba4799d52a-kube-api-access-wrvcw\") pod \"f0d73dfb-3a62-475d-ae3b-01ba4799d52a\" (UID: \"f0d73dfb-3a62-475d-ae3b-01ba4799d52a\") " Jul 09 04:18:31.686810 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:31.686764 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0d73dfb-3a62-475d-ae3b-01ba4799d52a-extensions-socket-volume\") pod \"f0d73dfb-3a62-475d-ae3b-01ba4799d52a\" (UID: \"f0d73dfb-3a62-475d-ae3b-01ba4799d52a\") " Jul 09 04:18:31.687263 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:31.687236 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0d73dfb-3a62-475d-ae3b-01ba4799d52a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f0d73dfb-3a62-475d-ae3b-01ba4799d52a" (UID: "f0d73dfb-3a62-475d-ae3b-01ba4799d52a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:18:31.688840 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:31.688820 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0d73dfb-3a62-475d-ae3b-01ba4799d52a-kube-api-access-wrvcw" (OuterVolumeSpecName: "kube-api-access-wrvcw") pod "f0d73dfb-3a62-475d-ae3b-01ba4799d52a" (UID: "f0d73dfb-3a62-475d-ae3b-01ba4799d52a"). InnerVolumeSpecName "kube-api-access-wrvcw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:18:31.787979 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:31.787948 2534 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0d73dfb-3a62-475d-ae3b-01ba4799d52a-extensions-socket-volume\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:18:31.787979 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:31.787973 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrvcw\" (UniqueName: \"kubernetes.io/projected/f0d73dfb-3a62-475d-ae3b-01ba4799d52a-kube-api-access-wrvcw\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:18:32.110001 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:32.109962 2534 generic.go:356] "Generic (PLEG): container finished" podID="f0d73dfb-3a62-475d-ae3b-01ba4799d52a" containerID="1804a3f640edecf2e8bfd17e45e7939ea896daa2e9fef97926de3f0ef2fbc41a" exitCode=0 Jul 09 04:18:32.110234 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:32.110028 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9v5n7" Jul 09 04:18:32.110234 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:32.110044 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9v5n7" event={"ID":"f0d73dfb-3a62-475d-ae3b-01ba4799d52a","Type":"ContainerDied","Data":"1804a3f640edecf2e8bfd17e45e7939ea896daa2e9fef97926de3f0ef2fbc41a"} Jul 09 04:18:32.110234 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:32.110120 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9v5n7" event={"ID":"f0d73dfb-3a62-475d-ae3b-01ba4799d52a","Type":"ContainerDied","Data":"1553b03d30c98d9d2ebd7567eca21a0d5b63b42ba268f630be9e92d513e983a9"} Jul 09 04:18:32.110234 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:32.110144 2534 scope.go:117] "RemoveContainer" containerID="1804a3f640edecf2e8bfd17e45e7939ea896daa2e9fef97926de3f0ef2fbc41a" Jul 09 04:18:32.119689 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:32.119669 2534 scope.go:117] "RemoveContainer" containerID="1804a3f640edecf2e8bfd17e45e7939ea896daa2e9fef97926de3f0ef2fbc41a" Jul 09 04:18:32.119941 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:18:32.119921 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1804a3f640edecf2e8bfd17e45e7939ea896daa2e9fef97926de3f0ef2fbc41a\": container with ID starting with 1804a3f640edecf2e8bfd17e45e7939ea896daa2e9fef97926de3f0ef2fbc41a not found: ID does not exist" containerID="1804a3f640edecf2e8bfd17e45e7939ea896daa2e9fef97926de3f0ef2fbc41a" Jul 09 04:18:32.120005 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:32.119953 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1804a3f640edecf2e8bfd17e45e7939ea896daa2e9fef97926de3f0ef2fbc41a"} err="failed to get container status \"1804a3f640edecf2e8bfd17e45e7939ea896daa2e9fef97926de3f0ef2fbc41a\": rpc error: code = NotFound desc = could not find container \"1804a3f640edecf2e8bfd17e45e7939ea896daa2e9fef97926de3f0ef2fbc41a\": container with ID starting with 1804a3f640edecf2e8bfd17e45e7939ea896daa2e9fef97926de3f0ef2fbc41a not found: ID does not exist" Jul 09 04:18:32.131955 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:32.131933 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9v5n7"] Jul 09 04:18:32.135862 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:32.135842 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9v5n7"] Jul 09 04:18:32.249975 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:32.249945 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0d73dfb-3a62-475d-ae3b-01ba4799d52a" path="/var/lib/kubelet/pods/f0d73dfb-3a62-475d-ae3b-01ba4799d52a/volumes" Jul 09 04:18:35.409254 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.409219 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s"] Jul 09 04:18:35.409807 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.409789 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0d73dfb-3a62-475d-ae3b-01ba4799d52a" containerName="manager" Jul 09 04:18:35.409807 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.409808 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0d73dfb-3a62-475d-ae3b-01ba4799d52a" containerName="manager" Jul 09 04:18:35.409917 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.409827 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d" containerName="manager" Jul 09 04:18:35.409917 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.409833 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d" containerName="manager" Jul 09 04:18:35.409917 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.409882 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="95cfd0ee-6cb3-450d-b3d6-58bdd8388a5d" containerName="manager" Jul 09 04:18:35.409917 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.409892 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0d73dfb-3a62-475d-ae3b-01ba4799d52a" containerName="manager" Jul 09 04:18:35.416461 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.416441 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.418735 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.418720 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-wk2wh\"" Jul 09 04:18:35.422344 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.422321 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s"] Jul 09 04:18:35.511503 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.511464 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/136f41c8-89e3-4c37-9bf8-e4d68b96a781-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.511503 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.511501 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/136f41c8-89e3-4c37-9bf8-e4d68b96a781-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.511732 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.511528 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2bf6\" (UniqueName: \"kubernetes.io/projected/136f41c8-89e3-4c37-9bf8-e4d68b96a781-kube-api-access-k2bf6\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.511732 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.511568 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/136f41c8-89e3-4c37-9bf8-e4d68b96a781-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.511732 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.511585 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/136f41c8-89e3-4c37-9bf8-e4d68b96a781-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.511732 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.511647 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/136f41c8-89e3-4c37-9bf8-e4d68b96a781-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.511732 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.511674 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/136f41c8-89e3-4c37-9bf8-e4d68b96a781-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.511732 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.511693 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/136f41c8-89e3-4c37-9bf8-e4d68b96a781-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.511732 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.511710 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/136f41c8-89e3-4c37-9bf8-e4d68b96a781-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.612445 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.612413 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/136f41c8-89e3-4c37-9bf8-e4d68b96a781-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.612618 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.612456 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/136f41c8-89e3-4c37-9bf8-e4d68b96a781-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.612618 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.612488 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2bf6\" (UniqueName: \"kubernetes.io/projected/136f41c8-89e3-4c37-9bf8-e4d68b96a781-kube-api-access-k2bf6\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.612618 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.612514 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/136f41c8-89e3-4c37-9bf8-e4d68b96a781-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.612618 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.612539 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/136f41c8-89e3-4c37-9bf8-e4d68b96a781-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.612618 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.612600 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/136f41c8-89e3-4c37-9bf8-e4d68b96a781-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.612876 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.612625 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/136f41c8-89e3-4c37-9bf8-e4d68b96a781-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.612876 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.612651 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/136f41c8-89e3-4c37-9bf8-e4d68b96a781-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.612876 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.612680 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/136f41c8-89e3-4c37-9bf8-e4d68b96a781-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.612876 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.612850 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/136f41c8-89e3-4c37-9bf8-e4d68b96a781-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.613068 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.612958 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/136f41c8-89e3-4c37-9bf8-e4d68b96a781-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.613068 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.612978 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/136f41c8-89e3-4c37-9bf8-e4d68b96a781-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.613068 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.613019 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/136f41c8-89e3-4c37-9bf8-e4d68b96a781-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.613377 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.613360 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/136f41c8-89e3-4c37-9bf8-e4d68b96a781-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.615321 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.615272 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/136f41c8-89e3-4c37-9bf8-e4d68b96a781-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.615439 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.615361 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/136f41c8-89e3-4c37-9bf8-e4d68b96a781-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.622238 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.622207 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2bf6\" (UniqueName: \"kubernetes.io/projected/136f41c8-89e3-4c37-9bf8-e4d68b96a781-kube-api-access-k2bf6\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.622395 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.622376 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/136f41c8-89e3-4c37-9bf8-e4d68b96a781-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-khl6s\" (UID: \"136f41c8-89e3-4c37-9bf8-e4d68b96a781\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.727929 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.727852 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:35.861233 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.861202 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s"] Jul 09 04:18:35.863584 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:18:35.863558 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod136f41c8_89e3_4c37_9bf8_e4d68b96a781.slice/crio-4d55a2a67cdd4d466b33e93acb3162152c8a4644ecc027f38f7542d3c4d81c3c WatchSource:0}: Error finding container 4d55a2a67cdd4d466b33e93acb3162152c8a4644ecc027f38f7542d3c4d81c3c: Status 404 returned error can't find the container with id 4d55a2a67cdd4d466b33e93acb3162152c8a4644ecc027f38f7542d3c4d81c3c Jul 09 04:18:35.866092 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.866056 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:18:35.866173 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.866124 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:18:35.866173 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:35.866154 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:18:36.132024 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:36.131992 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" event={"ID":"136f41c8-89e3-4c37-9bf8-e4d68b96a781","Type":"ContainerStarted","Data":"c3ad1c982ae81f46c368df1f2ca89525134fb7ed84b109723c154ba5904e6ac4"} Jul 09 04:18:36.132196 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:36.132046 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" event={"ID":"136f41c8-89e3-4c37-9bf8-e4d68b96a781","Type":"ContainerStarted","Data":"4d55a2a67cdd4d466b33e93acb3162152c8a4644ecc027f38f7542d3c4d81c3c"} Jul 09 04:18:36.148726 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:36.148657 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" podStartSLOduration=1.148638933 podStartE2EDuration="1.148638933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:18:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:18:36.148258992 +0000 UTC m=+706.371877480" watchObservedRunningTime="2026-07-09 04:18:36.148638933 +0000 UTC m=+706.372257424" Jul 09 04:18:36.728984 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:36.728943 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:36.734370 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:36.734346 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:37.136309 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:37.136258 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:37.137394 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:37.137377 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:18:37.774214 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:37.774177 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xrc7j"] Jul 09 04:18:37.778909 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:37.778886 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-xrc7j" Jul 09 04:18:37.781392 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:37.781372 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 04:18:37.781483 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:37.781372 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 09 04:18:37.782235 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:37.782219 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 04:18:37.782351 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:37.782238 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lk2bg\"" Jul 09 04:18:37.787488 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:37.787465 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xrc7j"] Jul 09 04:18:37.835340 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:37.835310 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqnxk\" (UniqueName: \"kubernetes.io/projected/1a5cb312-a628-4edb-a27d-e849192e3270-kube-api-access-sqnxk\") pod \"limitador-limitador-64c94fd64f-xrc7j\" (UID: \"1a5cb312-a628-4edb-a27d-e849192e3270\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xrc7j" Jul 09 04:18:37.835492 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:37.835385 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1a5cb312-a628-4edb-a27d-e849192e3270-config-file\") pod \"limitador-limitador-64c94fd64f-xrc7j\" (UID: \"1a5cb312-a628-4edb-a27d-e849192e3270\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xrc7j" Jul 09 04:18:37.867270 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:37.867239 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xrc7j"] Jul 09 04:18:37.935847 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:37.935823 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1a5cb312-a628-4edb-a27d-e849192e3270-config-file\") pod \"limitador-limitador-64c94fd64f-xrc7j\" (UID: \"1a5cb312-a628-4edb-a27d-e849192e3270\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xrc7j" Jul 09 04:18:37.936000 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:37.935888 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqnxk\" (UniqueName: \"kubernetes.io/projected/1a5cb312-a628-4edb-a27d-e849192e3270-kube-api-access-sqnxk\") pod \"limitador-limitador-64c94fd64f-xrc7j\" (UID: \"1a5cb312-a628-4edb-a27d-e849192e3270\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xrc7j" Jul 09 04:18:37.936466 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:37.936448 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1a5cb312-a628-4edb-a27d-e849192e3270-config-file\") pod \"limitador-limitador-64c94fd64f-xrc7j\" (UID: \"1a5cb312-a628-4edb-a27d-e849192e3270\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xrc7j" Jul 09 04:18:37.943965 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:37.943945 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqnxk\" (UniqueName: \"kubernetes.io/projected/1a5cb312-a628-4edb-a27d-e849192e3270-kube-api-access-sqnxk\") pod \"limitador-limitador-64c94fd64f-xrc7j\" (UID: \"1a5cb312-a628-4edb-a27d-e849192e3270\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xrc7j" Jul 09 04:18:38.091088 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:38.091062 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-xrc7j" Jul 09 04:18:38.239719 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:38.239696 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xrc7j"] Jul 09 04:18:38.241935 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:18:38.241911 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a5cb312_a628_4edb_a27d_e849192e3270.slice/crio-79f01f10c5a0db8c63a38c6ed48a841a3aed459806df1c110024f6c47377400a WatchSource:0}: Error finding container 79f01f10c5a0db8c63a38c6ed48a841a3aed459806df1c110024f6c47377400a: Status 404 returned error can't find the container with id 79f01f10c5a0db8c63a38c6ed48a841a3aed459806df1c110024f6c47377400a Jul 09 04:18:39.149532 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:39.149493 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-xrc7j" event={"ID":"1a5cb312-a628-4edb-a27d-e849192e3270","Type":"ContainerStarted","Data":"79f01f10c5a0db8c63a38c6ed48a841a3aed459806df1c110024f6c47377400a"} Jul 09 04:18:41.159873 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:41.159777 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-xrc7j" event={"ID":"1a5cb312-a628-4edb-a27d-e849192e3270","Type":"ContainerStarted","Data":"28e9a41264749e3afc2ef9cf993b7251478d07212a75837aafcb0aa4d76bfffe"} Jul 09 04:18:41.159873 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:41.159857 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-xrc7j" Jul 09 04:18:41.176561 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:41.176512 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-xrc7j" podStartSLOduration=1.573503379 podStartE2EDuration="4.176496199s" totalImagesPullingTime="2.60299282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:18:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:18:41.174161682 +0000 UTC m=+711.397780170" watchObservedRunningTime="2026-07-09 04:18:41.176496199 +0000 UTC m=+711.400114748" Jul 09 04:18:52.165674 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:52.165646 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-xrc7j" Jul 09 04:18:54.441103 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:54.441074 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xrc7j"] Jul 09 04:18:54.441487 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:54.441293 2534 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-xrc7j" podUID="1a5cb312-a628-4edb-a27d-e849192e3270" containerName="limitador" containerID="cri-o://28e9a41264749e3afc2ef9cf993b7251478d07212a75837aafcb0aa4d76bfffe" gracePeriod=30 Jul 09 04:18:55.225212 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:55.225178 2534 generic.go:356] "Generic (PLEG): container finished" podID="1a5cb312-a628-4edb-a27d-e849192e3270" containerID="28e9a41264749e3afc2ef9cf993b7251478d07212a75837aafcb0aa4d76bfffe" exitCode=0 Jul 09 04:18:55.225415 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:55.225258 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-xrc7j" event={"ID":"1a5cb312-a628-4edb-a27d-e849192e3270","Type":"ContainerDied","Data":"28e9a41264749e3afc2ef9cf993b7251478d07212a75837aafcb0aa4d76bfffe"} Jul 09 04:18:55.386894 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:55.386873 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-xrc7j" Jul 09 04:18:55.501469 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:55.501379 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1a5cb312-a628-4edb-a27d-e849192e3270-config-file\") pod \"1a5cb312-a628-4edb-a27d-e849192e3270\" (UID: \"1a5cb312-a628-4edb-a27d-e849192e3270\") " Jul 09 04:18:55.501816 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:55.501480 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqnxk\" (UniqueName: \"kubernetes.io/projected/1a5cb312-a628-4edb-a27d-e849192e3270-kube-api-access-sqnxk\") pod \"1a5cb312-a628-4edb-a27d-e849192e3270\" (UID: \"1a5cb312-a628-4edb-a27d-e849192e3270\") " Jul 09 04:18:55.501816 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:55.501776 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a5cb312-a628-4edb-a27d-e849192e3270-config-file" (OuterVolumeSpecName: "config-file") pod "1a5cb312-a628-4edb-a27d-e849192e3270" (UID: "1a5cb312-a628-4edb-a27d-e849192e3270"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:18:55.503614 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:55.503594 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a5cb312-a628-4edb-a27d-e849192e3270-kube-api-access-sqnxk" (OuterVolumeSpecName: "kube-api-access-sqnxk") pod "1a5cb312-a628-4edb-a27d-e849192e3270" (UID: "1a5cb312-a628-4edb-a27d-e849192e3270"). InnerVolumeSpecName "kube-api-access-sqnxk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:18:55.602767 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:55.602723 2534 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1a5cb312-a628-4edb-a27d-e849192e3270-config-file\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:18:55.602767 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:55.602764 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqnxk\" (UniqueName: \"kubernetes.io/projected/1a5cb312-a628-4edb-a27d-e849192e3270-kube-api-access-sqnxk\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:18:56.232249 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:56.232219 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-xrc7j" Jul 09 04:18:56.232249 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:56.232230 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-xrc7j" event={"ID":"1a5cb312-a628-4edb-a27d-e849192e3270","Type":"ContainerDied","Data":"79f01f10c5a0db8c63a38c6ed48a841a3aed459806df1c110024f6c47377400a"} Jul 09 04:18:56.232529 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:56.232318 2534 scope.go:117] "RemoveContainer" containerID="28e9a41264749e3afc2ef9cf993b7251478d07212a75837aafcb0aa4d76bfffe" Jul 09 04:18:56.257069 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:56.257045 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xrc7j"] Jul 09 04:18:56.258548 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:56.258528 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xrc7j"] Jul 09 04:18:58.249681 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:58.249651 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a5cb312-a628-4edb-a27d-e849192e3270" path="/var/lib/kubelet/pods/1a5cb312-a628-4edb-a27d-e849192e3270/volumes" Jul 09 04:18:58.862722 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:58.862693 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-4rdq8"] Jul 09 04:18:58.863091 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:58.863073 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a5cb312-a628-4edb-a27d-e849192e3270" containerName="limitador" Jul 09 04:18:58.863091 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:58.863088 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a5cb312-a628-4edb-a27d-e849192e3270" containerName="limitador" Jul 09 04:18:58.863241 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:58.863163 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a5cb312-a628-4edb-a27d-e849192e3270" containerName="limitador" Jul 09 04:18:58.867782 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:58.867758 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4rdq8" Jul 09 04:18:58.870518 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:58.870491 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 09 04:18:58.870644 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:58.870524 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-hz99f\"" Jul 09 04:18:58.870644 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:58.870539 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 09 04:18:58.871657 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:58.871638 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 09 04:18:58.876174 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:58.876154 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-4rdq8"] Jul 09 04:18:58.927735 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:58.927706 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5rhv\" (UniqueName: \"kubernetes.io/projected/983f5f4c-a908-4b8d-a24f-87f231e6b5dd-kube-api-access-r5rhv\") pod \"postgres-69fb4594cf-4rdq8\" (UID: \"983f5f4c-a908-4b8d-a24f-87f231e6b5dd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4rdq8" Jul 09 04:18:58.927912 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:58.927763 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/983f5f4c-a908-4b8d-a24f-87f231e6b5dd-data\") pod \"postgres-69fb4594cf-4rdq8\" (UID: \"983f5f4c-a908-4b8d-a24f-87f231e6b5dd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4rdq8" Jul 09 04:18:59.029076 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:59.029034 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5rhv\" (UniqueName: \"kubernetes.io/projected/983f5f4c-a908-4b8d-a24f-87f231e6b5dd-kube-api-access-r5rhv\") pod \"postgres-69fb4594cf-4rdq8\" (UID: \"983f5f4c-a908-4b8d-a24f-87f231e6b5dd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4rdq8" Jul 09 04:18:59.029593 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:59.029565 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/983f5f4c-a908-4b8d-a24f-87f231e6b5dd-data\") pod \"postgres-69fb4594cf-4rdq8\" (UID: \"983f5f4c-a908-4b8d-a24f-87f231e6b5dd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4rdq8" Jul 09 04:18:59.030179 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:59.030141 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/983f5f4c-a908-4b8d-a24f-87f231e6b5dd-data\") pod \"postgres-69fb4594cf-4rdq8\" (UID: \"983f5f4c-a908-4b8d-a24f-87f231e6b5dd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4rdq8" Jul 09 04:18:59.038355 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:59.038302 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5rhv\" (UniqueName: \"kubernetes.io/projected/983f5f4c-a908-4b8d-a24f-87f231e6b5dd-kube-api-access-r5rhv\") pod \"postgres-69fb4594cf-4rdq8\" (UID: \"983f5f4c-a908-4b8d-a24f-87f231e6b5dd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4rdq8" Jul 09 04:18:59.179737 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:59.179648 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4rdq8" Jul 09 04:18:59.314466 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:18:59.314442 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-4rdq8"] Jul 09 04:18:59.316463 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:18:59.316435 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod983f5f4c_a908_4b8d_a24f_87f231e6b5dd.slice/crio-c42d53a65c64d0ba127999d628d8b626897fdeab16daeac13327015054d97255 WatchSource:0}: Error finding container c42d53a65c64d0ba127999d628d8b626897fdeab16daeac13327015054d97255: Status 404 returned error can't find the container with id c42d53a65c64d0ba127999d628d8b626897fdeab16daeac13327015054d97255 Jul 09 04:19:00.257257 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:00.257220 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4rdq8" event={"ID":"983f5f4c-a908-4b8d-a24f-87f231e6b5dd","Type":"ContainerStarted","Data":"c42d53a65c64d0ba127999d628d8b626897fdeab16daeac13327015054d97255"} Jul 09 04:19:04.278767 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:04.278679 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4rdq8" event={"ID":"983f5f4c-a908-4b8d-a24f-87f231e6b5dd","Type":"ContainerStarted","Data":"f96e842d2543970ed0ccc025e9288f50bb8b9f64019de2f2ae999cc45ff94239"} Jul 09 04:19:04.279151 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:04.278842 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4rdq8" Jul 09 04:19:04.295654 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:04.295612 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4rdq8" podStartSLOduration=1.649965541 podStartE2EDuration="6.295597973s" totalImagesPullingTime="4.645632432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:18:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:19:04.293356955 +0000 UTC m=+734.516975443" watchObservedRunningTime="2026-07-09 04:19:04.295597973 +0000 UTC m=+734.519216460" Jul 09 04:19:10.315801 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:10.315772 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4rdq8" Jul 09 04:19:17.686941 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:17.686900 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5c8b548cdd-bdhwf"] Jul 09 04:19:17.697512 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:17.697469 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c8b548cdd-bdhwf" Jul 09 04:19:17.697758 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:17.697724 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c8b548cdd-bdhwf"] Jul 09 04:19:17.700067 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:17.700043 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 04:19:17.700188 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:17.700065 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-sq4lk\"" Jul 09 04:19:17.700257 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:17.700051 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 04:19:17.703317 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:17.703266 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx6t7\" (UniqueName: \"kubernetes.io/projected/189f68d1-c6bf-4b44-9861-9d4970d38e69-kube-api-access-vx6t7\") pod \"maas-controller-5c8b548cdd-bdhwf\" (UID: \"189f68d1-c6bf-4b44-9861-9d4970d38e69\") " pod="opendatahub/maas-controller-5c8b548cdd-bdhwf" Jul 09 04:19:17.703453 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:17.703434 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/189f68d1-c6bf-4b44-9861-9d4970d38e69-webhook-cert\") pod \"maas-controller-5c8b548cdd-bdhwf\" (UID: \"189f68d1-c6bf-4b44-9861-9d4970d38e69\") " pod="opendatahub/maas-controller-5c8b548cdd-bdhwf" Jul 09 04:19:17.804545 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:17.804507 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx6t7\" (UniqueName: \"kubernetes.io/projected/189f68d1-c6bf-4b44-9861-9d4970d38e69-kube-api-access-vx6t7\") pod \"maas-controller-5c8b548cdd-bdhwf\" (UID: \"189f68d1-c6bf-4b44-9861-9d4970d38e69\") " pod="opendatahub/maas-controller-5c8b548cdd-bdhwf" Jul 09 04:19:17.804722 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:17.804609 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/189f68d1-c6bf-4b44-9861-9d4970d38e69-webhook-cert\") pod \"maas-controller-5c8b548cdd-bdhwf\" (UID: \"189f68d1-c6bf-4b44-9861-9d4970d38e69\") " pod="opendatahub/maas-controller-5c8b548cdd-bdhwf" Jul 09 04:19:17.804772 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:19:17.804726 2534 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 09 04:19:17.804806 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:19:17.804798 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/189f68d1-c6bf-4b44-9861-9d4970d38e69-webhook-cert podName:189f68d1-c6bf-4b44-9861-9d4970d38e69 nodeName:}" failed. No retries permitted until 2026-07-09 04:19:18.304778052 +0000 UTC m=+748.528396518 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/189f68d1-c6bf-4b44-9861-9d4970d38e69-webhook-cert") pod "maas-controller-5c8b548cdd-bdhwf" (UID: "189f68d1-c6bf-4b44-9861-9d4970d38e69") : secret "maas-controller-webhook-cert" not found Jul 09 04:19:17.819176 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:17.819142 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx6t7\" (UniqueName: \"kubernetes.io/projected/189f68d1-c6bf-4b44-9861-9d4970d38e69-kube-api-access-vx6t7\") pod \"maas-controller-5c8b548cdd-bdhwf\" (UID: \"189f68d1-c6bf-4b44-9861-9d4970d38e69\") " pod="opendatahub/maas-controller-5c8b548cdd-bdhwf" Jul 09 04:19:17.868234 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:17.868201 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-f7cf9bf97-hjgm8"] Jul 09 04:19:17.872485 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:17.872462 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f7cf9bf97-hjgm8" Jul 09 04:19:17.881359 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:17.881335 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f7cf9bf97-hjgm8"] Jul 09 04:19:17.905429 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:17.905401 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx7n9\" (UniqueName: \"kubernetes.io/projected/1c2a924f-6d7f-475c-bc3e-224445e1a7a7-kube-api-access-cx7n9\") pod \"maas-controller-f7cf9bf97-hjgm8\" (UID: \"1c2a924f-6d7f-475c-bc3e-224445e1a7a7\") " pod="opendatahub/maas-controller-f7cf9bf97-hjgm8" Jul 09 04:19:17.905579 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:17.905543 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1c2a924f-6d7f-475c-bc3e-224445e1a7a7-webhook-cert\") pod \"maas-controller-f7cf9bf97-hjgm8\" (UID: \"1c2a924f-6d7f-475c-bc3e-224445e1a7a7\") " pod="opendatahub/maas-controller-f7cf9bf97-hjgm8" Jul 09 04:19:18.006599 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:18.006519 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1c2a924f-6d7f-475c-bc3e-224445e1a7a7-webhook-cert\") pod \"maas-controller-f7cf9bf97-hjgm8\" (UID: \"1c2a924f-6d7f-475c-bc3e-224445e1a7a7\") " pod="opendatahub/maas-controller-f7cf9bf97-hjgm8" Jul 09 04:19:18.006749 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:18.006600 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx7n9\" (UniqueName: \"kubernetes.io/projected/1c2a924f-6d7f-475c-bc3e-224445e1a7a7-kube-api-access-cx7n9\") pod \"maas-controller-f7cf9bf97-hjgm8\" (UID: \"1c2a924f-6d7f-475c-bc3e-224445e1a7a7\") " pod="opendatahub/maas-controller-f7cf9bf97-hjgm8" Jul 09 04:19:18.009107 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:18.009083 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1c2a924f-6d7f-475c-bc3e-224445e1a7a7-webhook-cert\") pod \"maas-controller-f7cf9bf97-hjgm8\" (UID: \"1c2a924f-6d7f-475c-bc3e-224445e1a7a7\") " pod="opendatahub/maas-controller-f7cf9bf97-hjgm8" Jul 09 04:19:18.015123 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:18.015103 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx7n9\" (UniqueName: \"kubernetes.io/projected/1c2a924f-6d7f-475c-bc3e-224445e1a7a7-kube-api-access-cx7n9\") pod \"maas-controller-f7cf9bf97-hjgm8\" (UID: \"1c2a924f-6d7f-475c-bc3e-224445e1a7a7\") " pod="opendatahub/maas-controller-f7cf9bf97-hjgm8" Jul 09 04:19:18.183889 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:18.183862 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f7cf9bf97-hjgm8" Jul 09 04:19:18.309415 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:18.309386 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/189f68d1-c6bf-4b44-9861-9d4970d38e69-webhook-cert\") pod \"maas-controller-5c8b548cdd-bdhwf\" (UID: \"189f68d1-c6bf-4b44-9861-9d4970d38e69\") " pod="opendatahub/maas-controller-5c8b548cdd-bdhwf" Jul 09 04:19:18.311851 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:18.311832 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/189f68d1-c6bf-4b44-9861-9d4970d38e69-webhook-cert\") pod \"maas-controller-5c8b548cdd-bdhwf\" (UID: \"189f68d1-c6bf-4b44-9861-9d4970d38e69\") " pod="opendatahub/maas-controller-5c8b548cdd-bdhwf" Jul 09 04:19:18.319263 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:18.319245 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f7cf9bf97-hjgm8"] Jul 09 04:19:18.321066 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:19:18.321044 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c2a924f_6d7f_475c_bc3e_224445e1a7a7.slice/crio-91314f370ccfa59e4907865303767ff1bf23f21981d2ae5ccef3aba7b8fef262 WatchSource:0}: Error finding container 91314f370ccfa59e4907865303767ff1bf23f21981d2ae5ccef3aba7b8fef262: Status 404 returned error can't find the container with id 91314f370ccfa59e4907865303767ff1bf23f21981d2ae5ccef3aba7b8fef262 Jul 09 04:19:18.349741 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:18.349713 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f7cf9bf97-hjgm8" event={"ID":"1c2a924f-6d7f-475c-bc3e-224445e1a7a7","Type":"ContainerStarted","Data":"91314f370ccfa59e4907865303767ff1bf23f21981d2ae5ccef3aba7b8fef262"} Jul 09 04:19:18.608498 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:18.608467 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c8b548cdd-bdhwf" Jul 09 04:19:18.746403 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:18.746368 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c8b548cdd-bdhwf"] Jul 09 04:19:18.749921 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:19:18.749895 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod189f68d1_c6bf_4b44_9861_9d4970d38e69.slice/crio-4c458861fad3ac714312f11afa033881906a0c87421aa37d65c205a63242b767 WatchSource:0}: Error finding container 4c458861fad3ac714312f11afa033881906a0c87421aa37d65c205a63242b767: Status 404 returned error can't find the container with id 4c458861fad3ac714312f11afa033881906a0c87421aa37d65c205a63242b767 Jul 09 04:19:19.359711 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:19.359667 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c8b548cdd-bdhwf" event={"ID":"189f68d1-c6bf-4b44-9861-9d4970d38e69","Type":"ContainerStarted","Data":"4c458861fad3ac714312f11afa033881906a0c87421aa37d65c205a63242b767"} Jul 09 04:19:22.379386 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:22.379338 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c8b548cdd-bdhwf" event={"ID":"189f68d1-c6bf-4b44-9861-9d4970d38e69","Type":"ContainerStarted","Data":"c797e70b3233b405000a6c973df57df08b0d78b9a352067fade3ebe6d9d1d457"} Jul 09 04:19:22.379851 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:22.379560 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5c8b548cdd-bdhwf" Jul 09 04:19:22.381011 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:22.380983 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f7cf9bf97-hjgm8" event={"ID":"1c2a924f-6d7f-475c-bc3e-224445e1a7a7","Type":"ContainerStarted","Data":"9e9bde93c716e27597eef033e1233bccffef1fea229da1986698b003bd9aeaef"} Jul 09 04:19:22.381160 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:22.381145 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-f7cf9bf97-hjgm8" Jul 09 04:19:22.396133 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:22.396083 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5c8b548cdd-bdhwf" podStartSLOduration=2.8548137909999998 podStartE2EDuration="5.396066005s" totalImagesPullingTime="2.541252214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:19:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:19:22.394524413 +0000 UTC m=+752.618142918" watchObservedRunningTime="2026-07-09 04:19:22.396066005 +0000 UTC m=+752.619684493" Jul 09 04:19:22.410217 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:22.410165 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-f7cf9bf97-hjgm8" podStartSLOduration=2.428220143 podStartE2EDuration="5.410149591s" totalImagesPullingTime="2.981929448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:19:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:19:22.408307652 +0000 UTC m=+752.631926143" watchObservedRunningTime="2026-07-09 04:19:22.410149591 +0000 UTC m=+752.633768093" Jul 09 04:19:33.388344 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:33.388317 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-f7cf9bf97-hjgm8" Jul 09 04:19:33.388739 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:33.388391 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5c8b548cdd-bdhwf" Jul 09 04:19:33.431821 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:33.431257 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c8b548cdd-bdhwf"] Jul 09 04:19:33.435647 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:33.435614 2534 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-5c8b548cdd-bdhwf" podUID="189f68d1-c6bf-4b44-9861-9d4970d38e69" containerName="manager" containerID="cri-o://c797e70b3233b405000a6c973df57df08b0d78b9a352067fade3ebe6d9d1d457" gracePeriod=30 Jul 09 04:19:33.684549 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:33.684527 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c8b548cdd-bdhwf" Jul 09 04:19:33.759196 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:33.759161 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/189f68d1-c6bf-4b44-9861-9d4970d38e69-webhook-cert\") pod \"189f68d1-c6bf-4b44-9861-9d4970d38e69\" (UID: \"189f68d1-c6bf-4b44-9861-9d4970d38e69\") " Jul 09 04:19:33.759389 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:33.759208 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vx6t7\" (UniqueName: \"kubernetes.io/projected/189f68d1-c6bf-4b44-9861-9d4970d38e69-kube-api-access-vx6t7\") pod \"189f68d1-c6bf-4b44-9861-9d4970d38e69\" (UID: \"189f68d1-c6bf-4b44-9861-9d4970d38e69\") " Jul 09 04:19:33.761562 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:33.761538 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/189f68d1-c6bf-4b44-9861-9d4970d38e69-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "189f68d1-c6bf-4b44-9861-9d4970d38e69" (UID: "189f68d1-c6bf-4b44-9861-9d4970d38e69"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:19:33.761621 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:33.761542 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/189f68d1-c6bf-4b44-9861-9d4970d38e69-kube-api-access-vx6t7" (OuterVolumeSpecName: "kube-api-access-vx6t7") pod "189f68d1-c6bf-4b44-9861-9d4970d38e69" (UID: "189f68d1-c6bf-4b44-9861-9d4970d38e69"). InnerVolumeSpecName "kube-api-access-vx6t7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:19:33.860722 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:33.860688 2534 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/189f68d1-c6bf-4b44-9861-9d4970d38e69-webhook-cert\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:19:33.860722 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:33.860720 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vx6t7\" (UniqueName: \"kubernetes.io/projected/189f68d1-c6bf-4b44-9861-9d4970d38e69-kube-api-access-vx6t7\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:19:34.442466 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:34.442429 2534 generic.go:356] "Generic (PLEG): container finished" podID="189f68d1-c6bf-4b44-9861-9d4970d38e69" containerID="c797e70b3233b405000a6c973df57df08b0d78b9a352067fade3ebe6d9d1d457" exitCode=0 Jul 09 04:19:34.442466 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:34.442469 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c8b548cdd-bdhwf" event={"ID":"189f68d1-c6bf-4b44-9861-9d4970d38e69","Type":"ContainerDied","Data":"c797e70b3233b405000a6c973df57df08b0d78b9a352067fade3ebe6d9d1d457"} Jul 09 04:19:34.442964 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:34.442491 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c8b548cdd-bdhwf" Jul 09 04:19:34.442964 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:34.442511 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c8b548cdd-bdhwf" event={"ID":"189f68d1-c6bf-4b44-9861-9d4970d38e69","Type":"ContainerDied","Data":"4c458861fad3ac714312f11afa033881906a0c87421aa37d65c205a63242b767"} Jul 09 04:19:34.442964 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:34.442531 2534 scope.go:117] "RemoveContainer" containerID="c797e70b3233b405000a6c973df57df08b0d78b9a352067fade3ebe6d9d1d457" Jul 09 04:19:34.451149 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:34.451123 2534 scope.go:117] "RemoveContainer" containerID="c797e70b3233b405000a6c973df57df08b0d78b9a352067fade3ebe6d9d1d457" Jul 09 04:19:34.451443 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:19:34.451422 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c797e70b3233b405000a6c973df57df08b0d78b9a352067fade3ebe6d9d1d457\": container with ID starting with c797e70b3233b405000a6c973df57df08b0d78b9a352067fade3ebe6d9d1d457 not found: ID does not exist" containerID="c797e70b3233b405000a6c973df57df08b0d78b9a352067fade3ebe6d9d1d457" Jul 09 04:19:34.451498 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:34.451451 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c797e70b3233b405000a6c973df57df08b0d78b9a352067fade3ebe6d9d1d457"} err="failed to get container status \"c797e70b3233b405000a6c973df57df08b0d78b9a352067fade3ebe6d9d1d457\": rpc error: code = NotFound desc = could not find container \"c797e70b3233b405000a6c973df57df08b0d78b9a352067fade3ebe6d9d1d457\": container with ID starting with c797e70b3233b405000a6c973df57df08b0d78b9a352067fade3ebe6d9d1d457 not found: ID does not exist" Jul 09 04:19:34.458783 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:34.458758 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c8b548cdd-bdhwf"] Jul 09 04:19:34.460460 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:34.460441 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5c8b548cdd-bdhwf"] Jul 09 04:19:36.251204 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:36.251175 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="189f68d1-c6bf-4b44-9861-9d4970d38e69" path="/var/lib/kubelet/pods/189f68d1-c6bf-4b44-9861-9d4970d38e69/volumes" Jul 09 04:19:54.582532 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:54.582501 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5b664864d4-ls5q8"] Jul 09 04:19:54.583077 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:54.582897 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="189f68d1-c6bf-4b44-9861-9d4970d38e69" containerName="manager" Jul 09 04:19:54.583077 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:54.582909 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="189f68d1-c6bf-4b44-9861-9d4970d38e69" containerName="manager" Jul 09 04:19:54.583077 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:54.583002 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="189f68d1-c6bf-4b44-9861-9d4970d38e69" containerName="manager" Jul 09 04:19:54.600671 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:54.600637 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b664864d4-ls5q8"] Jul 09 04:19:54.600829 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:54.600751 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5b664864d4-ls5q8" Jul 09 04:19:54.603598 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:54.603573 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 09 04:19:54.603737 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:54.603630 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-b2ppv\"" Jul 09 04:19:54.655676 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:54.655646 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59a544dc-35c1-4fb9-9b8f-d30d9c268d60-openshift-service-ca\") pod \"maas-api-5b664864d4-ls5q8\" (UID: \"59a544dc-35c1-4fb9-9b8f-d30d9c268d60\") " pod="odh-ai-gateway-infra/maas-api-5b664864d4-ls5q8" Jul 09 04:19:54.655837 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:54.655716 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbpcj\" (UniqueName: \"kubernetes.io/projected/59a544dc-35c1-4fb9-9b8f-d30d9c268d60-kube-api-access-tbpcj\") pod \"maas-api-5b664864d4-ls5q8\" (UID: \"59a544dc-35c1-4fb9-9b8f-d30d9c268d60\") " pod="odh-ai-gateway-infra/maas-api-5b664864d4-ls5q8" Jul 09 04:19:54.655837 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:54.655797 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59a544dc-35c1-4fb9-9b8f-d30d9c268d60-maas-api-tls\") pod \"maas-api-5b664864d4-ls5q8\" (UID: \"59a544dc-35c1-4fb9-9b8f-d30d9c268d60\") " pod="odh-ai-gateway-infra/maas-api-5b664864d4-ls5q8" Jul 09 04:19:54.756750 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:54.756708 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59a544dc-35c1-4fb9-9b8f-d30d9c268d60-maas-api-tls\") pod \"maas-api-5b664864d4-ls5q8\" (UID: \"59a544dc-35c1-4fb9-9b8f-d30d9c268d60\") " pod="odh-ai-gateway-infra/maas-api-5b664864d4-ls5q8" Jul 09 04:19:54.756941 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:54.756775 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59a544dc-35c1-4fb9-9b8f-d30d9c268d60-openshift-service-ca\") pod \"maas-api-5b664864d4-ls5q8\" (UID: \"59a544dc-35c1-4fb9-9b8f-d30d9c268d60\") " pod="odh-ai-gateway-infra/maas-api-5b664864d4-ls5q8" Jul 09 04:19:54.756941 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:19:54.756860 2534 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 09 04:19:54.756941 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:54.756891 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbpcj\" (UniqueName: \"kubernetes.io/projected/59a544dc-35c1-4fb9-9b8f-d30d9c268d60-kube-api-access-tbpcj\") pod \"maas-api-5b664864d4-ls5q8\" (UID: \"59a544dc-35c1-4fb9-9b8f-d30d9c268d60\") " pod="odh-ai-gateway-infra/maas-api-5b664864d4-ls5q8" Jul 09 04:19:54.756941 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:19:54.756928 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59a544dc-35c1-4fb9-9b8f-d30d9c268d60-maas-api-tls podName:59a544dc-35c1-4fb9-9b8f-d30d9c268d60 nodeName:}" failed. No retries permitted until 2026-07-09 04:19:55.256912275 +0000 UTC m=+785.480530741 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/59a544dc-35c1-4fb9-9b8f-d30d9c268d60-maas-api-tls") pod "maas-api-5b664864d4-ls5q8" (UID: "59a544dc-35c1-4fb9-9b8f-d30d9c268d60") : secret "maas-api-serving-cert" not found Jul 09 04:19:54.757475 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:54.757454 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59a544dc-35c1-4fb9-9b8f-d30d9c268d60-openshift-service-ca\") pod \"maas-api-5b664864d4-ls5q8\" (UID: \"59a544dc-35c1-4fb9-9b8f-d30d9c268d60\") " pod="odh-ai-gateway-infra/maas-api-5b664864d4-ls5q8" Jul 09 04:19:54.765731 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:54.765712 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbpcj\" (UniqueName: \"kubernetes.io/projected/59a544dc-35c1-4fb9-9b8f-d30d9c268d60-kube-api-access-tbpcj\") pod \"maas-api-5b664864d4-ls5q8\" (UID: \"59a544dc-35c1-4fb9-9b8f-d30d9c268d60\") " pod="odh-ai-gateway-infra/maas-api-5b664864d4-ls5q8" Jul 09 04:19:55.096962 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.096925 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-v9b9w"] Jul 09 04:19:55.099447 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.099427 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-v9b9w" Jul 09 04:19:55.101985 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.101964 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-tkrx7\"" Jul 09 04:19:55.102105 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.101989 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 09 04:19:55.111375 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.111351 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-v9b9w"] Jul 09 04:19:55.113007 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.112985 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-prdjf"] Jul 09 04:19:55.117343 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.117326 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-prdjf" Jul 09 04:19:55.127434 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.127413 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-prdjf"] Jul 09 04:19:55.159999 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.159973 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfsk6\" (UniqueName: \"kubernetes.io/projected/54b0094b-edbc-4be5-8c85-b6dbed9171cf-kube-api-access-cfsk6\") pod \"payload-pre-processing-7964457696-v9b9w\" (UID: \"54b0094b-edbc-4be5-8c85-b6dbed9171cf\") " pod="openshift-ingress/payload-pre-processing-7964457696-v9b9w" Jul 09 04:19:55.160153 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.160009 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr8zr\" (UniqueName: \"kubernetes.io/projected/51c51e26-4ee8-4759-9885-2d7c389c662d-kube-api-access-vr8zr\") pod \"payload-processing-58f86c7798-prdjf\" (UID: \"51c51e26-4ee8-4759-9885-2d7c389c662d\") " pod="openshift-ingress/payload-processing-58f86c7798-prdjf" Jul 09 04:19:55.160153 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.160057 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54b0094b-edbc-4be5-8c85-b6dbed9171cf-plugins-config-volume\") pod \"payload-pre-processing-7964457696-v9b9w\" (UID: \"54b0094b-edbc-4be5-8c85-b6dbed9171cf\") " pod="openshift-ingress/payload-pre-processing-7964457696-v9b9w" Jul 09 04:19:55.160153 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.160109 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51c51e26-4ee8-4759-9885-2d7c389c662d-plugins-config-volume\") pod \"payload-processing-58f86c7798-prdjf\" (UID: \"51c51e26-4ee8-4759-9885-2d7c389c662d\") " pod="openshift-ingress/payload-processing-58f86c7798-prdjf" Jul 09 04:19:55.261140 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.261057 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfsk6\" (UniqueName: \"kubernetes.io/projected/54b0094b-edbc-4be5-8c85-b6dbed9171cf-kube-api-access-cfsk6\") pod \"payload-pre-processing-7964457696-v9b9w\" (UID: \"54b0094b-edbc-4be5-8c85-b6dbed9171cf\") " pod="openshift-ingress/payload-pre-processing-7964457696-v9b9w" Jul 09 04:19:55.261812 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.261782 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr8zr\" (UniqueName: \"kubernetes.io/projected/51c51e26-4ee8-4759-9885-2d7c389c662d-kube-api-access-vr8zr\") pod \"payload-processing-58f86c7798-prdjf\" (UID: \"51c51e26-4ee8-4759-9885-2d7c389c662d\") " pod="openshift-ingress/payload-processing-58f86c7798-prdjf" Jul 09 04:19:55.262265 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.262242 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54b0094b-edbc-4be5-8c85-b6dbed9171cf-plugins-config-volume\") pod \"payload-pre-processing-7964457696-v9b9w\" (UID: \"54b0094b-edbc-4be5-8c85-b6dbed9171cf\") " pod="openshift-ingress/payload-pre-processing-7964457696-v9b9w" Jul 09 04:19:55.262447 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.262411 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51c51e26-4ee8-4759-9885-2d7c389c662d-plugins-config-volume\") pod \"payload-processing-58f86c7798-prdjf\" (UID: \"51c51e26-4ee8-4759-9885-2d7c389c662d\") " pod="openshift-ingress/payload-processing-58f86c7798-prdjf" Jul 09 04:19:55.262649 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.262460 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59a544dc-35c1-4fb9-9b8f-d30d9c268d60-maas-api-tls\") pod \"maas-api-5b664864d4-ls5q8\" (UID: \"59a544dc-35c1-4fb9-9b8f-d30d9c268d60\") " pod="odh-ai-gateway-infra/maas-api-5b664864d4-ls5q8" Jul 09 04:19:55.263171 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.263148 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54b0094b-edbc-4be5-8c85-b6dbed9171cf-plugins-config-volume\") pod \"payload-pre-processing-7964457696-v9b9w\" (UID: \"54b0094b-edbc-4be5-8c85-b6dbed9171cf\") " pod="openshift-ingress/payload-pre-processing-7964457696-v9b9w" Jul 09 04:19:55.263419 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.263395 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51c51e26-4ee8-4759-9885-2d7c389c662d-plugins-config-volume\") pod \"payload-processing-58f86c7798-prdjf\" (UID: \"51c51e26-4ee8-4759-9885-2d7c389c662d\") " pod="openshift-ingress/payload-processing-58f86c7798-prdjf" Jul 09 04:19:55.265430 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.265411 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59a544dc-35c1-4fb9-9b8f-d30d9c268d60-maas-api-tls\") pod \"maas-api-5b664864d4-ls5q8\" (UID: \"59a544dc-35c1-4fb9-9b8f-d30d9c268d60\") " pod="odh-ai-gateway-infra/maas-api-5b664864d4-ls5q8" Jul 09 04:19:55.270014 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.269996 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfsk6\" (UniqueName: \"kubernetes.io/projected/54b0094b-edbc-4be5-8c85-b6dbed9171cf-kube-api-access-cfsk6\") pod \"payload-pre-processing-7964457696-v9b9w\" (UID: \"54b0094b-edbc-4be5-8c85-b6dbed9171cf\") " pod="openshift-ingress/payload-pre-processing-7964457696-v9b9w" Jul 09 04:19:55.270118 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.270082 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr8zr\" (UniqueName: \"kubernetes.io/projected/51c51e26-4ee8-4759-9885-2d7c389c662d-kube-api-access-vr8zr\") pod \"payload-processing-58f86c7798-prdjf\" (UID: \"51c51e26-4ee8-4759-9885-2d7c389c662d\") " pod="openshift-ingress/payload-processing-58f86c7798-prdjf" Jul 09 04:19:55.411349 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.411242 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-v9b9w" Jul 09 04:19:55.430379 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.430349 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-prdjf" Jul 09 04:19:55.512763 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.512702 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5b664864d4-ls5q8" Jul 09 04:19:55.563822 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.563800 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-v9b9w"] Jul 09 04:19:55.567412 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:19:55.567364 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54b0094b_edbc_4be5_8c85_b6dbed9171cf.slice/crio-bc374a36803d5f8fd106a2dc2605112828fb3e4e3fb12f27ea4145344aaa29d4 WatchSource:0}: Error finding container bc374a36803d5f8fd106a2dc2605112828fb3e4e3fb12f27ea4145344aaa29d4: Status 404 returned error can't find the container with id bc374a36803d5f8fd106a2dc2605112828fb3e4e3fb12f27ea4145344aaa29d4 Jul 09 04:19:55.590577 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.590555 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-prdjf"] Jul 09 04:19:55.594793 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:19:55.594765 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51c51e26_4ee8_4759_9885_2d7c389c662d.slice/crio-0dda4960d7dbbf56f1f1621368c595915b8706665037277de33c30848c3a3ed8 WatchSource:0}: Error finding container 0dda4960d7dbbf56f1f1621368c595915b8706665037277de33c30848c3a3ed8: Status 404 returned error can't find the container with id 0dda4960d7dbbf56f1f1621368c595915b8706665037277de33c30848c3a3ed8 Jul 09 04:19:55.668300 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:55.668256 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b664864d4-ls5q8"] Jul 09 04:19:55.670801 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:19:55.670767 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59a544dc_35c1_4fb9_9b8f_d30d9c268d60.slice/crio-17345c4ed9ba4237c29dc7ac2e93e36509dac87ab8fce843b4fc7edf20511487 WatchSource:0}: Error finding container 17345c4ed9ba4237c29dc7ac2e93e36509dac87ab8fce843b4fc7edf20511487: Status 404 returned error can't find the container with id 17345c4ed9ba4237c29dc7ac2e93e36509dac87ab8fce843b4fc7edf20511487 Jul 09 04:19:56.552611 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:56.552573 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-prdjf" event={"ID":"51c51e26-4ee8-4759-9885-2d7c389c662d","Type":"ContainerStarted","Data":"0dda4960d7dbbf56f1f1621368c595915b8706665037277de33c30848c3a3ed8"} Jul 09 04:19:56.554681 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:56.554648 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5b664864d4-ls5q8" event={"ID":"59a544dc-35c1-4fb9-9b8f-d30d9c268d60","Type":"ContainerStarted","Data":"17345c4ed9ba4237c29dc7ac2e93e36509dac87ab8fce843b4fc7edf20511487"} Jul 09 04:19:56.556245 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:56.556218 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-v9b9w" event={"ID":"54b0094b-edbc-4be5-8c85-b6dbed9171cf","Type":"ContainerStarted","Data":"bc374a36803d5f8fd106a2dc2605112828fb3e4e3fb12f27ea4145344aaa29d4"} Jul 09 04:19:57.563826 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:57.563790 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-v9b9w" event={"ID":"54b0094b-edbc-4be5-8c85-b6dbed9171cf","Type":"ContainerStarted","Data":"819cde2933fe6669d82bc1546eb8dcdec9a06fb4186efc8976e2cc71794484f3"} Jul 09 04:19:57.564477 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:57.564459 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-v9b9w" Jul 09 04:19:57.565650 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:57.565631 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-prdjf" event={"ID":"51c51e26-4ee8-4759-9885-2d7c389c662d","Type":"ContainerStarted","Data":"253ea43afcf83a179b6d7f827f28b7fe1ca6f2a3ff17989afe61803d9dfcb4b5"} Jul 09 04:19:57.566023 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:57.565992 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-prdjf" Jul 09 04:19:57.580549 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:57.580490 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-v9b9w" podStartSLOduration=0.899320912 podStartE2EDuration="2.580475334s" totalImagesPullingTime="1.681154422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:19:57.578085363 +0000 UTC m=+787.801703851" watchObservedRunningTime="2026-07-09 04:19:57.580475334 +0000 UTC m=+787.804093821" Jul 09 04:19:57.592251 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:57.592205 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-prdjf" podStartSLOduration=0.936041423 podStartE2EDuration="2.592193565s" totalImagesPullingTime="1.656152142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:19:57.591638676 +0000 UTC m=+787.815257165" watchObservedRunningTime="2026-07-09 04:19:57.592193565 +0000 UTC m=+787.815812053" Jul 09 04:19:58.571903 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:58.571859 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5b664864d4-ls5q8" event={"ID":"59a544dc-35c1-4fb9-9b8f-d30d9c268d60","Type":"ContainerStarted","Data":"9f6cf92473f85ea45393746ddc7ea0f511ece621600adbda6a3e3358d64ea60b"} Jul 09 04:19:58.572347 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:58.572185 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5b664864d4-ls5q8" Jul 09 04:19:58.590194 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:19:58.590142 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5b664864d4-ls5q8" podStartSLOduration=2.407111144 podStartE2EDuration="4.590129244s" totalImagesPullingTime="2.1830181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:19:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:19:58.58727667 +0000 UTC m=+788.810895168" watchObservedRunningTime="2026-07-09 04:19:58.590129244 +0000 UTC m=+788.813747731" Jul 09 04:20:03.598938 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:03.598903 2534 generic.go:356] "Generic (PLEG): container finished" podID="136f41c8-89e3-4c37-9bf8-e4d68b96a781" containerID="c3ad1c982ae81f46c368df1f2ca89525134fb7ed84b109723c154ba5904e6ac4" exitCode=0 Jul 09 04:20:03.599386 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:03.598986 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" event={"ID":"136f41c8-89e3-4c37-9bf8-e4d68b96a781","Type":"ContainerDied","Data":"c3ad1c982ae81f46c368df1f2ca89525134fb7ed84b109723c154ba5904e6ac4"} Jul 09 04:20:03.599434 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:03.599423 2534 scope.go:117] "RemoveContainer" containerID="c3ad1c982ae81f46c368df1f2ca89525134fb7ed84b109723c154ba5904e6ac4" Jul 09 04:20:03.600470 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:03.600435 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:20:03.600566 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:03.600507 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:20:03.600566 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:03.600540 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:20:04.583096 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:04.583066 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5b664864d4-ls5q8" Jul 09 04:20:04.606833 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:04.606797 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" event={"ID":"136f41c8-89e3-4c37-9bf8-e4d68b96a781","Type":"ContainerStarted","Data":"6a72f34fe9adb1e4c792c95fd9b79aa0bfd5c389969af221e79ee203ff71a9d2"} Jul 09 04:20:04.728470 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:04.728426 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:20:04.730169 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:04.730128 2534 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-khl6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 09 04:20:04.730357 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:04.730198 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" podUID="136f41c8-89e3-4c37-9bf8-e4d68b96a781" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 09 04:20:05.728571 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:05.728536 2534 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-khl6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 09 04:20:05.729161 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:05.729131 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" podUID="136f41c8-89e3-4c37-9bf8-e4d68b96a781" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 09 04:20:06.729145 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:06.729107 2534 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-khl6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 09 04:20:06.729816 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:06.729786 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" podUID="136f41c8-89e3-4c37-9bf8-e4d68b96a781" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 09 04:20:07.136630 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:07.136588 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:20:07.729158 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:07.729113 2534 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-khl6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 09 04:20:07.729679 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:07.729183 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" podUID="136f41c8-89e3-4c37-9bf8-e4d68b96a781" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 09 04:20:08.728349 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:08.728309 2534 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-khl6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 09 04:20:08.728538 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:08.728366 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" podUID="136f41c8-89e3-4c37-9bf8-e4d68b96a781" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 09 04:20:09.579478 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:09.579438 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-v9b9w" Jul 09 04:20:09.579972 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:09.579515 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-prdjf" Jul 09 04:20:09.732230 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:09.732192 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:20:09.733367 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:09.733346 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-khl6s" Jul 09 04:20:16.754934 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:16.754899 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2"] Jul 09 04:20:16.757970 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:16.757951 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" Jul 09 04:20:16.761528 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:16.761511 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-hnnjh\"" Jul 09 04:20:16.761657 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:16.761530 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 09 04:20:16.761657 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:16.761512 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 09 04:20:16.761657 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:16.761621 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 09 04:20:16.767354 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:16.767331 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2"] Jul 09 04:20:16.872021 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:16.871989 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9236344a-f135-4188-919c-59061b63d8e3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2\" (UID: \"9236344a-f135-4188-919c-59061b63d8e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" Jul 09 04:20:16.872021 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:16.872028 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9236344a-f135-4188-919c-59061b63d8e3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2\" (UID: \"9236344a-f135-4188-919c-59061b63d8e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" Jul 09 04:20:16.872227 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:16.872076 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9236344a-f135-4188-919c-59061b63d8e3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2\" (UID: \"9236344a-f135-4188-919c-59061b63d8e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" Jul 09 04:20:16.872227 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:16.872100 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9236344a-f135-4188-919c-59061b63d8e3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2\" (UID: \"9236344a-f135-4188-919c-59061b63d8e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" Jul 09 04:20:16.872227 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:16.872119 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j6l5\" (UniqueName: \"kubernetes.io/projected/9236344a-f135-4188-919c-59061b63d8e3-kube-api-access-8j6l5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2\" (UID: \"9236344a-f135-4188-919c-59061b63d8e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" Jul 09 04:20:16.872227 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:16.872203 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9236344a-f135-4188-919c-59061b63d8e3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2\" (UID: \"9236344a-f135-4188-919c-59061b63d8e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" Jul 09 04:20:16.973300 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:16.973249 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9236344a-f135-4188-919c-59061b63d8e3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2\" (UID: \"9236344a-f135-4188-919c-59061b63d8e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" Jul 09 04:20:16.973466 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:16.973317 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9236344a-f135-4188-919c-59061b63d8e3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2\" (UID: \"9236344a-f135-4188-919c-59061b63d8e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" Jul 09 04:20:16.973466 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:16.973349 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j6l5\" (UniqueName: \"kubernetes.io/projected/9236344a-f135-4188-919c-59061b63d8e3-kube-api-access-8j6l5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2\" (UID: \"9236344a-f135-4188-919c-59061b63d8e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" Jul 09 04:20:16.973466 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:16.973391 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9236344a-f135-4188-919c-59061b63d8e3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2\" (UID: \"9236344a-f135-4188-919c-59061b63d8e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" Jul 09 04:20:16.973628 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:16.973465 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9236344a-f135-4188-919c-59061b63d8e3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2\" (UID: \"9236344a-f135-4188-919c-59061b63d8e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" Jul 09 04:20:16.973628 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:16.973500 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9236344a-f135-4188-919c-59061b63d8e3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2\" (UID: \"9236344a-f135-4188-919c-59061b63d8e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" Jul 09 04:20:16.973873 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:16.973848 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9236344a-f135-4188-919c-59061b63d8e3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2\" (UID: \"9236344a-f135-4188-919c-59061b63d8e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" Jul 09 04:20:16.973944 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:16.973873 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9236344a-f135-4188-919c-59061b63d8e3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2\" (UID: \"9236344a-f135-4188-919c-59061b63d8e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" Jul 09 04:20:16.973944 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:16.973860 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9236344a-f135-4188-919c-59061b63d8e3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2\" (UID: \"9236344a-f135-4188-919c-59061b63d8e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" Jul 09 04:20:16.975957 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:16.975932 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9236344a-f135-4188-919c-59061b63d8e3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2\" (UID: \"9236344a-f135-4188-919c-59061b63d8e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" Jul 09 04:20:16.976121 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:16.976099 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9236344a-f135-4188-919c-59061b63d8e3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2\" (UID: \"9236344a-f135-4188-919c-59061b63d8e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" Jul 09 04:20:16.986681 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:16.986662 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j6l5\" (UniqueName: \"kubernetes.io/projected/9236344a-f135-4188-919c-59061b63d8e3-kube-api-access-8j6l5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2\" (UID: \"9236344a-f135-4188-919c-59061b63d8e3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" Jul 09 04:20:17.069011 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:17.068984 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" Jul 09 04:20:17.216471 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:17.216442 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2"] Jul 09 04:20:17.217860 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:20:17.217828 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9236344a_f135_4188_919c_59061b63d8e3.slice/crio-47e5f1661b29c6d6051b85ee7a357edbed2cd3c3bec253e7b27ad3171406c1a6 WatchSource:0}: Error finding container 47e5f1661b29c6d6051b85ee7a357edbed2cd3c3bec253e7b27ad3171406c1a6: Status 404 returned error can't find the container with id 47e5f1661b29c6d6051b85ee7a357edbed2cd3c3bec253e7b27ad3171406c1a6 Jul 09 04:20:17.683518 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:17.683438 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" event={"ID":"9236344a-f135-4188-919c-59061b63d8e3","Type":"ContainerStarted","Data":"47e5f1661b29c6d6051b85ee7a357edbed2cd3c3bec253e7b27ad3171406c1a6"} Jul 09 04:20:18.689836 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:18.689757 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" event={"ID":"9236344a-f135-4188-919c-59061b63d8e3","Type":"ContainerStarted","Data":"23de5d5277d9d0a4be7b81d0fce11c74d64e50bc4e0f1c4084bfb12c2262f5a2"} Jul 09 04:20:18.706932 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:18.706883 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" podStartSLOduration=1.5620197359999999 podStartE2EDuration="2.70686783s" totalImagesPullingTime="1.144848094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:20:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:20:18.70492469 +0000 UTC m=+808.928543188" watchObservedRunningTime="2026-07-09 04:20:18.70686783 +0000 UTC m=+808.930486377" Jul 09 04:20:27.069919 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:27.069883 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" Jul 09 04:20:27.070324 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:27.069955 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" Jul 09 04:20:27.082555 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:27.082534 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" Jul 09 04:20:27.746410 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:27.746368 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2" Jul 09 04:20:30.848864 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:30.848828 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc"] Jul 09 04:20:30.852274 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:30.852252 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" Jul 09 04:20:30.854586 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:30.854562 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 09 04:20:30.863687 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:30.863664 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc"] Jul 09 04:20:31.009010 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:31.008979 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5c62ba3-ff83-42ee-9deb-e3396f084dda-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mz8hc\" (UID: \"c5c62ba3-ff83-42ee-9deb-e3396f084dda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" Jul 09 04:20:31.009010 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:31.009015 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5c62ba3-ff83-42ee-9deb-e3396f084dda-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mz8hc\" (UID: \"c5c62ba3-ff83-42ee-9deb-e3396f084dda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" Jul 09 04:20:31.009255 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:31.009034 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5c62ba3-ff83-42ee-9deb-e3396f084dda-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mz8hc\" (UID: \"c5c62ba3-ff83-42ee-9deb-e3396f084dda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" Jul 09 04:20:31.009255 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:31.009114 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5c62ba3-ff83-42ee-9deb-e3396f084dda-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mz8hc\" (UID: \"c5c62ba3-ff83-42ee-9deb-e3396f084dda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" Jul 09 04:20:31.009255 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:31.009181 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5c62ba3-ff83-42ee-9deb-e3396f084dda-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mz8hc\" (UID: \"c5c62ba3-ff83-42ee-9deb-e3396f084dda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" Jul 09 04:20:31.009255 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:31.009215 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmkgn\" (UniqueName: \"kubernetes.io/projected/c5c62ba3-ff83-42ee-9deb-e3396f084dda-kube-api-access-fmkgn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mz8hc\" (UID: \"c5c62ba3-ff83-42ee-9deb-e3396f084dda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" Jul 09 04:20:31.110673 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:31.110588 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5c62ba3-ff83-42ee-9deb-e3396f084dda-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mz8hc\" (UID: \"c5c62ba3-ff83-42ee-9deb-e3396f084dda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" Jul 09 04:20:31.110673 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:31.110643 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmkgn\" (UniqueName: \"kubernetes.io/projected/c5c62ba3-ff83-42ee-9deb-e3396f084dda-kube-api-access-fmkgn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mz8hc\" (UID: \"c5c62ba3-ff83-42ee-9deb-e3396f084dda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" Jul 09 04:20:31.110921 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:31.110688 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5c62ba3-ff83-42ee-9deb-e3396f084dda-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mz8hc\" (UID: \"c5c62ba3-ff83-42ee-9deb-e3396f084dda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" Jul 09 04:20:31.110921 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:31.110708 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5c62ba3-ff83-42ee-9deb-e3396f084dda-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mz8hc\" (UID: \"c5c62ba3-ff83-42ee-9deb-e3396f084dda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" Jul 09 04:20:31.110921 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:31.110725 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5c62ba3-ff83-42ee-9deb-e3396f084dda-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mz8hc\" (UID: \"c5c62ba3-ff83-42ee-9deb-e3396f084dda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" Jul 09 04:20:31.110921 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:31.110756 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5c62ba3-ff83-42ee-9deb-e3396f084dda-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mz8hc\" (UID: \"c5c62ba3-ff83-42ee-9deb-e3396f084dda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" Jul 09 04:20:31.111170 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:31.111135 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5c62ba3-ff83-42ee-9deb-e3396f084dda-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mz8hc\" (UID: \"c5c62ba3-ff83-42ee-9deb-e3396f084dda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" Jul 09 04:20:31.111170 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:31.111160 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5c62ba3-ff83-42ee-9deb-e3396f084dda-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mz8hc\" (UID: \"c5c62ba3-ff83-42ee-9deb-e3396f084dda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" Jul 09 04:20:31.111260 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:31.111196 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5c62ba3-ff83-42ee-9deb-e3396f084dda-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mz8hc\" (UID: \"c5c62ba3-ff83-42ee-9deb-e3396f084dda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" Jul 09 04:20:31.113164 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:31.113141 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5c62ba3-ff83-42ee-9deb-e3396f084dda-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mz8hc\" (UID: \"c5c62ba3-ff83-42ee-9deb-e3396f084dda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" Jul 09 04:20:31.113349 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:31.113331 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5c62ba3-ff83-42ee-9deb-e3396f084dda-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mz8hc\" (UID: \"c5c62ba3-ff83-42ee-9deb-e3396f084dda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" Jul 09 04:20:31.118478 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:31.118458 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmkgn\" (UniqueName: \"kubernetes.io/projected/c5c62ba3-ff83-42ee-9deb-e3396f084dda-kube-api-access-fmkgn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mz8hc\" (UID: \"c5c62ba3-ff83-42ee-9deb-e3396f084dda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" Jul 09 04:20:31.163157 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:31.163123 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" Jul 09 04:20:31.299768 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:31.299741 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc"] Jul 09 04:20:31.302423 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:20:31.302397 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5c62ba3_ff83_42ee_9deb_e3396f084dda.slice/crio-a8f827102f61c60a6b82a7fb9819765c00d74a9949df05a98a282a1ebcf20ebc WatchSource:0}: Error finding container a8f827102f61c60a6b82a7fb9819765c00d74a9949df05a98a282a1ebcf20ebc: Status 404 returned error can't find the container with id a8f827102f61c60a6b82a7fb9819765c00d74a9949df05a98a282a1ebcf20ebc Jul 09 04:20:31.755974 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:31.755881 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" event={"ID":"c5c62ba3-ff83-42ee-9deb-e3396f084dda","Type":"ContainerStarted","Data":"7a77b8e785d722bf5f8245a1aa9f999a81decc554fd18d89f14faaa812ccba17"} Jul 09 04:20:31.755974 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:31.755940 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" event={"ID":"c5c62ba3-ff83-42ee-9deb-e3396f084dda","Type":"ContainerStarted","Data":"a8f827102f61c60a6b82a7fb9819765c00d74a9949df05a98a282a1ebcf20ebc"} Jul 09 04:20:41.163553 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:41.163505 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" Jul 09 04:20:41.164022 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:41.163584 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" Jul 09 04:20:41.176099 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:41.176076 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" Jul 09 04:20:41.193025 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:41.192972 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" podStartSLOduration=11.044291303 podStartE2EDuration="11.192959627s" totalImagesPullingTime="148.668324ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:20:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:20:31.772246948 +0000 UTC m=+821.995865436" watchObservedRunningTime="2026-07-09 04:20:41.192959627 +0000 UTC m=+831.416578114" Jul 09 04:20:41.815463 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:20:41.815432 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mz8hc" Jul 09 04:21:50.228644 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:21:50.228566 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8dnq_633a3ae5-3d78-4217-901a-6d0d06442cf4/console-operator/1.log" Jul 09 04:21:50.229134 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:21:50.228785 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8dnq_633a3ae5-3d78-4217-901a-6d0d06442cf4/console-operator/1.log" Jul 09 04:23:24.274658 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:24.274625 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f7cf9bf97-hjgm8"] Jul 09 04:23:24.275313 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:24.274802 2534 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-f7cf9bf97-hjgm8" podUID="1c2a924f-6d7f-475c-bc3e-224445e1a7a7" containerName="manager" containerID="cri-o://9e9bde93c716e27597eef033e1233bccffef1fea229da1986698b003bd9aeaef" gracePeriod=30 Jul 09 04:23:24.516499 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:24.516477 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f7cf9bf97-hjgm8" Jul 09 04:23:24.598856 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:24.598819 2534 generic.go:356] "Generic (PLEG): container finished" podID="1c2a924f-6d7f-475c-bc3e-224445e1a7a7" containerID="9e9bde93c716e27597eef033e1233bccffef1fea229da1986698b003bd9aeaef" exitCode=0 Jul 09 04:23:24.599040 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:24.598864 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f7cf9bf97-hjgm8" Jul 09 04:23:24.599040 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:24.598897 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f7cf9bf97-hjgm8" event={"ID":"1c2a924f-6d7f-475c-bc3e-224445e1a7a7","Type":"ContainerDied","Data":"9e9bde93c716e27597eef033e1233bccffef1fea229da1986698b003bd9aeaef"} Jul 09 04:23:24.599040 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:24.598951 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f7cf9bf97-hjgm8" event={"ID":"1c2a924f-6d7f-475c-bc3e-224445e1a7a7","Type":"ContainerDied","Data":"91314f370ccfa59e4907865303767ff1bf23f21981d2ae5ccef3aba7b8fef262"} Jul 09 04:23:24.599040 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:24.598974 2534 scope.go:117] "RemoveContainer" containerID="9e9bde93c716e27597eef033e1233bccffef1fea229da1986698b003bd9aeaef" Jul 09 04:23:24.607519 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:24.607490 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1c2a924f-6d7f-475c-bc3e-224445e1a7a7-webhook-cert\") pod \"1c2a924f-6d7f-475c-bc3e-224445e1a7a7\" (UID: \"1c2a924f-6d7f-475c-bc3e-224445e1a7a7\") " Jul 09 04:23:24.607656 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:24.607609 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cx7n9\" (UniqueName: \"kubernetes.io/projected/1c2a924f-6d7f-475c-bc3e-224445e1a7a7-kube-api-access-cx7n9\") pod \"1c2a924f-6d7f-475c-bc3e-224445e1a7a7\" (UID: \"1c2a924f-6d7f-475c-bc3e-224445e1a7a7\") " Jul 09 04:23:24.608216 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:24.608153 2534 scope.go:117] "RemoveContainer" containerID="9e9bde93c716e27597eef033e1233bccffef1fea229da1986698b003bd9aeaef" Jul 09 04:23:24.608526 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:23:24.608502 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e9bde93c716e27597eef033e1233bccffef1fea229da1986698b003bd9aeaef\": container with ID starting with 9e9bde93c716e27597eef033e1233bccffef1fea229da1986698b003bd9aeaef not found: ID does not exist" containerID="9e9bde93c716e27597eef033e1233bccffef1fea229da1986698b003bd9aeaef" Jul 09 04:23:24.608647 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:24.608529 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e9bde93c716e27597eef033e1233bccffef1fea229da1986698b003bd9aeaef"} err="failed to get container status \"9e9bde93c716e27597eef033e1233bccffef1fea229da1986698b003bd9aeaef\": rpc error: code = NotFound desc = could not find container \"9e9bde93c716e27597eef033e1233bccffef1fea229da1986698b003bd9aeaef\": container with ID starting with 9e9bde93c716e27597eef033e1233bccffef1fea229da1986698b003bd9aeaef not found: ID does not exist" Jul 09 04:23:24.609632 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:24.609611 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c2a924f-6d7f-475c-bc3e-224445e1a7a7-kube-api-access-cx7n9" (OuterVolumeSpecName: "kube-api-access-cx7n9") pod "1c2a924f-6d7f-475c-bc3e-224445e1a7a7" (UID: "1c2a924f-6d7f-475c-bc3e-224445e1a7a7"). InnerVolumeSpecName "kube-api-access-cx7n9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:23:24.609715 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:24.609628 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c2a924f-6d7f-475c-bc3e-224445e1a7a7-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "1c2a924f-6d7f-475c-bc3e-224445e1a7a7" (UID: "1c2a924f-6d7f-475c-bc3e-224445e1a7a7"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:23:24.709104 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:24.709074 2534 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1c2a924f-6d7f-475c-bc3e-224445e1a7a7-webhook-cert\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:23:24.709104 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:24.709106 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cx7n9\" (UniqueName: \"kubernetes.io/projected/1c2a924f-6d7f-475c-bc3e-224445e1a7a7-kube-api-access-cx7n9\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:23:24.919698 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:24.919663 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f7cf9bf97-hjgm8"] Jul 09 04:23:24.923242 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:24.923220 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-f7cf9bf97-hjgm8"] Jul 09 04:23:25.667984 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:25.667951 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-f7cf9bf97-pmb2x"] Jul 09 04:23:25.668443 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:25.668427 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c2a924f-6d7f-475c-bc3e-224445e1a7a7" containerName="manager" Jul 09 04:23:25.668503 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:25.668445 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c2a924f-6d7f-475c-bc3e-224445e1a7a7" containerName="manager" Jul 09 04:23:25.668550 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:25.668540 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c2a924f-6d7f-475c-bc3e-224445e1a7a7" containerName="manager" Jul 09 04:23:25.672723 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:25.672708 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f7cf9bf97-pmb2x" Jul 09 04:23:25.675046 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:25.675025 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 04:23:25.676074 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:25.676050 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 04:23:25.676074 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:25.676059 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-sq4lk\"" Jul 09 04:23:25.679681 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:25.679657 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f7cf9bf97-pmb2x"] Jul 09 04:23:25.819570 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:25.819528 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkhnz\" (UniqueName: \"kubernetes.io/projected/01c7c3c4-6115-404b-815b-058927f121a1-kube-api-access-hkhnz\") pod \"maas-controller-f7cf9bf97-pmb2x\" (UID: \"01c7c3c4-6115-404b-815b-058927f121a1\") " pod="opendatahub/maas-controller-f7cf9bf97-pmb2x" Jul 09 04:23:25.819738 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:25.819612 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01c7c3c4-6115-404b-815b-058927f121a1-webhook-cert\") pod \"maas-controller-f7cf9bf97-pmb2x\" (UID: \"01c7c3c4-6115-404b-815b-058927f121a1\") " pod="opendatahub/maas-controller-f7cf9bf97-pmb2x" Jul 09 04:23:25.920205 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:25.920131 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01c7c3c4-6115-404b-815b-058927f121a1-webhook-cert\") pod \"maas-controller-f7cf9bf97-pmb2x\" (UID: \"01c7c3c4-6115-404b-815b-058927f121a1\") " pod="opendatahub/maas-controller-f7cf9bf97-pmb2x" Jul 09 04:23:25.920380 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:25.920214 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkhnz\" (UniqueName: \"kubernetes.io/projected/01c7c3c4-6115-404b-815b-058927f121a1-kube-api-access-hkhnz\") pod \"maas-controller-f7cf9bf97-pmb2x\" (UID: \"01c7c3c4-6115-404b-815b-058927f121a1\") " pod="opendatahub/maas-controller-f7cf9bf97-pmb2x" Jul 09 04:23:25.922641 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:25.922617 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01c7c3c4-6115-404b-815b-058927f121a1-webhook-cert\") pod \"maas-controller-f7cf9bf97-pmb2x\" (UID: \"01c7c3c4-6115-404b-815b-058927f121a1\") " pod="opendatahub/maas-controller-f7cf9bf97-pmb2x" Jul 09 04:23:25.927688 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:25.927654 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkhnz\" (UniqueName: \"kubernetes.io/projected/01c7c3c4-6115-404b-815b-058927f121a1-kube-api-access-hkhnz\") pod \"maas-controller-f7cf9bf97-pmb2x\" (UID: \"01c7c3c4-6115-404b-815b-058927f121a1\") " pod="opendatahub/maas-controller-f7cf9bf97-pmb2x" Jul 09 04:23:25.984769 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:25.984743 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f7cf9bf97-pmb2x" Jul 09 04:23:26.115357 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:26.115335 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f7cf9bf97-pmb2x"] Jul 09 04:23:26.117610 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:23:26.117583 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01c7c3c4_6115_404b_815b_058927f121a1.slice/crio-34a09e5cb078b73dd55e183d0abe2c503a1f51b66f961cedf078a74aa81fcf3c WatchSource:0}: Error finding container 34a09e5cb078b73dd55e183d0abe2c503a1f51b66f961cedf078a74aa81fcf3c: Status 404 returned error can't find the container with id 34a09e5cb078b73dd55e183d0abe2c503a1f51b66f961cedf078a74aa81fcf3c Jul 09 04:23:26.118997 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:26.118978 2534 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 04:23:26.250421 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:26.250346 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c2a924f-6d7f-475c-bc3e-224445e1a7a7" path="/var/lib/kubelet/pods/1c2a924f-6d7f-475c-bc3e-224445e1a7a7/volumes" Jul 09 04:23:26.612543 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:26.612511 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f7cf9bf97-pmb2x" event={"ID":"01c7c3c4-6115-404b-815b-058927f121a1","Type":"ContainerStarted","Data":"85bb18a548a997ef81a610a17c8be3381d8c3d7f3c3b56454c322bcf5894cd32"} Jul 09 04:23:26.612644 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:26.612571 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f7cf9bf97-pmb2x" event={"ID":"01c7c3c4-6115-404b-815b-058927f121a1","Type":"ContainerStarted","Data":"34a09e5cb078b73dd55e183d0abe2c503a1f51b66f961cedf078a74aa81fcf3c"} Jul 09 04:23:26.612644 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:26.612599 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-f7cf9bf97-pmb2x" Jul 09 04:23:26.629607 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:26.629556 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-f7cf9bf97-pmb2x" podStartSLOduration=1.208289036 podStartE2EDuration="1.629535308s" totalImagesPullingTime="421.246272ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:23:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:23:26.626459565 +0000 UTC m=+996.850078051" watchObservedRunningTime="2026-07-09 04:23:26.629535308 +0000 UTC m=+996.853153796" Jul 09 04:23:37.620639 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:23:37.620613 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-f7cf9bf97-pmb2x" Jul 09 04:26:50.264934 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:26:50.264893 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8dnq_633a3ae5-3d78-4217-901a-6d0d06442cf4/console-operator/1.log" Jul 09 04:26:50.267512 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:26:50.265039 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8dnq_633a3ae5-3d78-4217-901a-6d0d06442cf4/console-operator/1.log" Jul 09 04:30:00.142792 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:30:00.142756 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726190-cb6pc"] Jul 09 04:30:00.146188 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:30:00.146171 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726190-cb6pc" Jul 09 04:30:00.148617 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:30:00.148596 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7qtgv\"" Jul 09 04:30:00.152999 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:30:00.152976 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726190-cb6pc"] Jul 09 04:30:00.157655 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:30:00.157633 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726190-cb6pc" Jul 09 04:30:00.312116 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:30:00.312085 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726190-cb6pc"] Jul 09 04:30:00.314435 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:30:00.314409 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eb4ac7a_4a12_4304_98b7_c1f1b95061a3.slice/crio-5942588b9095a969d40555f5ada83b5df6bc59727df53c8164f892fb4502f7d2 WatchSource:0}: Error finding container 5942588b9095a969d40555f5ada83b5df6bc59727df53c8164f892fb4502f7d2: Status 404 returned error can't find the container with id 5942588b9095a969d40555f5ada83b5df6bc59727df53c8164f892fb4502f7d2 Jul 09 04:30:00.316311 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:30:00.316298 2534 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 04:30:00.591345 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:30:00.591314 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726190-cb6pc" event={"ID":"1eb4ac7a-4a12-4304-98b7-c1f1b95061a3","Type":"ContainerStarted","Data":"5942588b9095a969d40555f5ada83b5df6bc59727df53c8164f892fb4502f7d2"} Jul 09 04:30:03.609443 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:30:03.609404 2534 generic.go:356] "Generic (PLEG): container finished" podID="1eb4ac7a-4a12-4304-98b7-c1f1b95061a3" containerID="e71e6b0fb183aff738119d53681c1f7c94d4948b7ba7cfafd9cc822e8869407a" exitCode=0 Jul 09 04:30:03.609814 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:30:03.609491 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726190-cb6pc" event={"ID":"1eb4ac7a-4a12-4304-98b7-c1f1b95061a3","Type":"ContainerDied","Data":"e71e6b0fb183aff738119d53681c1f7c94d4948b7ba7cfafd9cc822e8869407a"} Jul 09 04:30:04.755623 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:30:04.755592 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726190-cb6pc" Jul 09 04:30:05.620728 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:30:05.620690 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726190-cb6pc" event={"ID":"1eb4ac7a-4a12-4304-98b7-c1f1b95061a3","Type":"ContainerDied","Data":"5942588b9095a969d40555f5ada83b5df6bc59727df53c8164f892fb4502f7d2"} Jul 09 04:30:05.620728 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:30:05.620719 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726190-cb6pc" Jul 09 04:30:05.620956 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:30:05.620737 2534 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5942588b9095a969d40555f5ada83b5df6bc59727df53c8164f892fb4502f7d2" Jul 09 04:31:50.300922 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:31:50.300898 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8dnq_633a3ae5-3d78-4217-901a-6d0d06442cf4/console-operator/1.log" Jul 09 04:31:50.303856 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:31:50.303835 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8dnq_633a3ae5-3d78-4217-901a-6d0d06442cf4/console-operator/1.log" Jul 09 04:35:35.351608 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.351571 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89"] Jul 09 04:35:35.352022 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.351982 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1eb4ac7a-4a12-4304-98b7-c1f1b95061a3" containerName="cleanup" Jul 09 04:35:35.352022 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.351993 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="1eb4ac7a-4a12-4304-98b7-c1f1b95061a3" containerName="cleanup" Jul 09 04:35:35.352100 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.352064 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="1eb4ac7a-4a12-4304-98b7-c1f1b95061a3" containerName="cleanup" Jul 09 04:35:35.354079 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.354063 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.359779 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.359755 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-51ad45e2-openshift-default-dockercfg-m57t8\"" Jul 09 04:35:35.427736 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.427702 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfnvr\" (UniqueName: \"kubernetes.io/projected/439b293c-fc3a-4ab8-a6c9-16e57815cf49-kube-api-access-qfnvr\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.427898 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.427741 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-workload-certs\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.427898 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.427772 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istio-podinfo\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.427898 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.427837 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istio-token\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.428002 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.427904 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-credential-socket\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.428002 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.427938 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istio-envoy\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.428002 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.427955 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istio-data\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.428002 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.427976 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istiod-ca-cert\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.428002 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.427992 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-workload-socket\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.457243 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.457212 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89"] Jul 09 04:35:35.529100 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.529064 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-credential-socket\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.529320 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.529122 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istio-envoy\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.529320 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.529150 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istio-data\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.529320 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.529182 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istiod-ca-cert\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.529320 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.529206 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-workload-socket\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.529320 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.529318 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfnvr\" (UniqueName: \"kubernetes.io/projected/439b293c-fc3a-4ab8-a6c9-16e57815cf49-kube-api-access-qfnvr\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.529613 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.529359 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-workload-certs\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.529613 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.529398 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istio-podinfo\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.529613 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.529438 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istio-token\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.529613 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.529513 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-credential-socket\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.529613 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.529566 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istio-data\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.529837 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.529694 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-workload-socket\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.529837 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.529793 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-workload-certs\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.529932 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.529911 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istiod-ca-cert\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.531610 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.531593 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istio-envoy\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.531905 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.531888 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istio-podinfo\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.544332 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.543846 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istio-token\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.549171 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.545033 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfnvr\" (UniqueName: \"kubernetes.io/projected/439b293c-fc3a-4ab8-a6c9-16e57815cf49-kube-api-access-qfnvr\") pod \"e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.665167 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.665128 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:35.803676 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.803649 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89"] Jul 09 04:35:35.805708 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:35:35.805676 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod439b293c_fc3a_4ab8_a6c9_16e57815cf49.slice/crio-5bf14438ed116307a507fe4a3ebd29d6b93d5d1c6b6a062c83d8c36fa274c6fa WatchSource:0}: Error finding container 5bf14438ed116307a507fe4a3ebd29d6b93d5d1c6b6a062c83d8c36fa274c6fa: Status 404 returned error can't find the container with id 5bf14438ed116307a507fe4a3ebd29d6b93d5d1c6b6a062c83d8c36fa274c6fa Jul 09 04:35:35.807924 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.807890 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:35:35.807978 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.807959 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:35:35.808021 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:35.808008 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:35:36.294411 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:36.294359 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" event={"ID":"439b293c-fc3a-4ab8-a6c9-16e57815cf49","Type":"ContainerStarted","Data":"ede17f42e542395577fa650399636c03940b6f513bed7166ffc2fae8735e9e7c"} Jul 09 04:35:36.294709 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:36.294429 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" event={"ID":"439b293c-fc3a-4ab8-a6c9-16e57815cf49","Type":"ContainerStarted","Data":"5bf14438ed116307a507fe4a3ebd29d6b93d5d1c6b6a062c83d8c36fa274c6fa"} Jul 09 04:35:36.317434 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:36.317383 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" podStartSLOduration=1.317363114 podStartE2EDuration="1.317363114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:35:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:35:36.315012494 +0000 UTC m=+1726.538630973" watchObservedRunningTime="2026-07-09 04:35:36.317363114 +0000 UTC m=+1726.540981603" Jul 09 04:35:36.576506 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:36.576422 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv"] Jul 09 04:35:36.579168 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:36.579149 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv" Jul 09 04:35:36.582632 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:36.582610 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-51ad45e2-serving-cert\"" Jul 09 04:35:36.591962 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:36.591941 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv"] Jul 09 04:35:36.666228 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:36.666193 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:36.680610 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:36.680581 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:36.742411 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:36.742378 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lsdp\" (UniqueName: \"kubernetes.io/projected/4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1-kube-api-access-7lsdp\") pod \"maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv\" (UID: \"4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv" Jul 09 04:35:36.742589 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:36.742448 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1-maas-api-tls\") pod \"maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv\" (UID: \"4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv" Jul 09 04:35:36.742589 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:36.742476 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1-openshift-service-ca\") pod \"maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv\" (UID: \"4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv" Jul 09 04:35:36.843811 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:36.843716 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1-maas-api-tls\") pod \"maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv\" (UID: \"4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv" Jul 09 04:35:36.843811 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:36.843762 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1-openshift-service-ca\") pod \"maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv\" (UID: \"4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv" Jul 09 04:35:36.844053 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:36.843836 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lsdp\" (UniqueName: \"kubernetes.io/projected/4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1-kube-api-access-7lsdp\") pod \"maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv\" (UID: \"4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv" Jul 09 04:35:36.844053 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:35:36.843861 2534 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-serving-cert: secret "maas-api-e2e-ait-51ad45e2-serving-cert" not found Jul 09 04:35:36.844053 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:35:36.843921 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1-maas-api-tls podName:4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1 nodeName:}" failed. No retries permitted until 2026-07-09 04:35:37.343904777 +0000 UTC m=+1727.567523243 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1-maas-api-tls") pod "maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv" (UID: "4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1") : secret "maas-api-e2e-ait-51ad45e2-serving-cert" not found Jul 09 04:35:36.844579 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:36.844554 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1-openshift-service-ca\") pod \"maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv\" (UID: \"4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv" Jul 09 04:35:36.853063 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:36.853040 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lsdp\" (UniqueName: \"kubernetes.io/projected/4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1-kube-api-access-7lsdp\") pod \"maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv\" (UID: \"4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv" Jul 09 04:35:37.300066 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:37.300032 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:37.300960 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:37.300939 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:37.349963 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:37.349924 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1-maas-api-tls\") pod \"maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv\" (UID: \"4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv" Jul 09 04:35:37.353161 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:37.353137 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1-maas-api-tls\") pod \"maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv\" (UID: \"4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv" Jul 09 04:35:37.492902 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:37.492870 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv" Jul 09 04:35:37.633742 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:37.633720 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv"] Jul 09 04:35:37.635937 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:35:37.635899 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bb1c3d3_824a_4ddb_a932_c1c9cd2206e1.slice/crio-2b89fc4c83cb090a153a38a04bb158c7e87e85bc59176a8dac714bfefba026d7 WatchSource:0}: Error finding container 2b89fc4c83cb090a153a38a04bb158c7e87e85bc59176a8dac714bfefba026d7: Status 404 returned error can't find the container with id 2b89fc4c83cb090a153a38a04bb158c7e87e85bc59176a8dac714bfefba026d7 Jul 09 04:35:37.637465 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:37.637446 2534 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 04:35:38.305355 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:38.305323 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv" event={"ID":"4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1","Type":"ContainerStarted","Data":"2b89fc4c83cb090a153a38a04bb158c7e87e85bc59176a8dac714bfefba026d7"} Jul 09 04:35:39.312164 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:39.312129 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv" event={"ID":"4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1","Type":"ContainerStarted","Data":"cdf00bec38eeb83f44f12ecaed7ca20bf3856300d25d8256c8724bad4025b602"} Jul 09 04:35:39.312605 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:39.312204 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv" Jul 09 04:35:39.331762 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:39.331713 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv" podStartSLOduration=2.496457355 podStartE2EDuration="3.331694303s" totalImagesPullingTime="835.236948ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:35:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:35:39.331219537 +0000 UTC m=+1729.554838029" watchObservedRunningTime="2026-07-09 04:35:39.331694303 +0000 UTC m=+1729.555312792" Jul 09 04:35:45.328475 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:45.328443 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv" Jul 09 04:35:47.367936 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:47.367905 2534 generic.go:356] "Generic (PLEG): container finished" podID="439b293c-fc3a-4ab8-a6c9-16e57815cf49" containerID="ede17f42e542395577fa650399636c03940b6f513bed7166ffc2fae8735e9e7c" exitCode=0 Jul 09 04:35:47.368378 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:47.367958 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" event={"ID":"439b293c-fc3a-4ab8-a6c9-16e57815cf49","Type":"ContainerDied","Data":"ede17f42e542395577fa650399636c03940b6f513bed7166ffc2fae8735e9e7c"} Jul 09 04:35:47.368465 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:47.368448 2534 scope.go:117] "RemoveContainer" containerID="ede17f42e542395577fa650399636c03940b6f513bed7166ffc2fae8735e9e7c" Jul 09 04:35:47.369610 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:47.369580 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:35:47.369684 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:47.369672 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:35:47.369720 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:47.369707 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:35:48.374298 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:48.374255 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" event={"ID":"439b293c-fc3a-4ab8-a6c9-16e57815cf49","Type":"ContainerStarted","Data":"4648dfa2a351449ad972df1982b89bb756cd0a95f23c5493aae211945623f5e1"} Jul 09 04:35:48.666102 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:48.666006 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:48.667401 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:48.667376 2534 patch_prober.go:28] interesting pod/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 09 04:35:48.667466 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:48.667428 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" podUID="439b293c-fc3a-4ab8-a6c9-16e57815cf49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 09 04:35:49.665688 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:49.665653 2534 patch_prober.go:28] interesting pod/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 09 04:35:49.666106 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:49.665710 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" podUID="439b293c-fc3a-4ab8-a6c9-16e57815cf49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 09 04:35:50.665563 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:50.665528 2534 patch_prober.go:28] interesting pod/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 09 04:35:50.665763 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:50.665588 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" podUID="439b293c-fc3a-4ab8-a6c9-16e57815cf49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 09 04:35:51.666550 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:51.666514 2534 patch_prober.go:28] interesting pod/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 09 04:35:51.666939 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:51.666591 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" podUID="439b293c-fc3a-4ab8-a6c9-16e57815cf49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 09 04:35:52.300988 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:52.300951 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:52.669802 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:52.669730 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:35:52.670842 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:35:52.670823 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:36:41.592159 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:41.592083 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89"] Jul 09 04:36:41.592683 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:41.592314 2534 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" podUID="439b293c-fc3a-4ab8-a6c9-16e57815cf49" containerName="istio-proxy" containerID="cri-o://4648dfa2a351449ad972df1982b89bb756cd0a95f23c5493aae211945623f5e1" gracePeriod=30 Jul 09 04:36:46.856102 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:46.856038 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:36:46.905304 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:46.905259 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qfnvr\" (UniqueName: \"kubernetes.io/projected/439b293c-fc3a-4ab8-a6c9-16e57815cf49-kube-api-access-qfnvr\") pod \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " Jul 09 04:36:46.905469 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:46.905387 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istio-envoy\") pod \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " Jul 09 04:36:46.905469 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:46.905428 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-workload-certs\") pod \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " Jul 09 04:36:46.905595 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:46.905479 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istio-data\") pod \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " Jul 09 04:36:46.905595 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:46.905514 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-credential-socket\") pod \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " Jul 09 04:36:46.905595 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:46.905554 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istio-podinfo\") pod \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " Jul 09 04:36:46.905595 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:46.905577 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istio-token\") pod \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " Jul 09 04:36:46.905804 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:46.905634 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-workload-socket\") pod \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " Jul 09 04:36:46.905804 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:46.905662 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istiod-ca-cert\") pod \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\" (UID: \"439b293c-fc3a-4ab8-a6c9-16e57815cf49\") " Jul 09 04:36:46.905906 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:46.905843 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "439b293c-fc3a-4ab8-a6c9-16e57815cf49" (UID: "439b293c-fc3a-4ab8-a6c9-16e57815cf49"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:36:46.906045 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:46.906010 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "439b293c-fc3a-4ab8-a6c9-16e57815cf49" (UID: "439b293c-fc3a-4ab8-a6c9-16e57815cf49"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:36:46.906236 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:46.906116 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "439b293c-fc3a-4ab8-a6c9-16e57815cf49" (UID: "439b293c-fc3a-4ab8-a6c9-16e57815cf49"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:36:46.906320 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:46.906247 2534 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-workload-socket\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:36:46.906320 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:46.906265 2534 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-workload-certs\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:36:46.906450 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:46.906334 2534 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-credential-socket\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:36:46.906501 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:46.906479 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "439b293c-fc3a-4ab8-a6c9-16e57815cf49" (UID: "439b293c-fc3a-4ab8-a6c9-16e57815cf49"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:36:46.906731 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:46.906691 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istio-data" (OuterVolumeSpecName: "istio-data") pod "439b293c-fc3a-4ab8-a6c9-16e57815cf49" (UID: "439b293c-fc3a-4ab8-a6c9-16e57815cf49"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:36:46.908626 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:46.908604 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "439b293c-fc3a-4ab8-a6c9-16e57815cf49" (UID: "439b293c-fc3a-4ab8-a6c9-16e57815cf49"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 04:36:46.908811 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:46.908794 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "439b293c-fc3a-4ab8-a6c9-16e57815cf49" (UID: "439b293c-fc3a-4ab8-a6c9-16e57815cf49"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:36:46.909011 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:46.908985 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/439b293c-fc3a-4ab8-a6c9-16e57815cf49-kube-api-access-qfnvr" (OuterVolumeSpecName: "kube-api-access-qfnvr") pod "439b293c-fc3a-4ab8-a6c9-16e57815cf49" (UID: "439b293c-fc3a-4ab8-a6c9-16e57815cf49"). InnerVolumeSpecName "kube-api-access-qfnvr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:36:46.909085 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:46.908993 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istio-token" (OuterVolumeSpecName: "istio-token") pod "439b293c-fc3a-4ab8-a6c9-16e57815cf49" (UID: "439b293c-fc3a-4ab8-a6c9-16e57815cf49"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:36:47.007666 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.007635 2534 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istio-data\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:36:47.007666 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.007665 2534 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istio-podinfo\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:36:47.007919 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.007677 2534 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istio-token\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:36:47.007919 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.007690 2534 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istiod-ca-cert\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:36:47.007919 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.007701 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfnvr\" (UniqueName: \"kubernetes.io/projected/439b293c-fc3a-4ab8-a6c9-16e57815cf49-kube-api-access-qfnvr\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:36:47.007919 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.007713 2534 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/439b293c-fc3a-4ab8-a6c9-16e57815cf49-istio-envoy\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:36:47.085394 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.085355 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv"] Jul 09 04:36:47.086349 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.086081 2534 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv" podUID="4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1" containerName="maas-api" containerID="cri-o://cdf00bec38eeb83f44f12ecaed7ca20bf3856300d25d8256c8724bad4025b602" gracePeriod=30 Jul 09 04:36:47.329340 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.329313 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv" Jul 09 04:36:47.412171 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.412138 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lsdp\" (UniqueName: \"kubernetes.io/projected/4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1-kube-api-access-7lsdp\") pod \"4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1\" (UID: \"4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1\") " Jul 09 04:36:47.412374 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.412199 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1-maas-api-tls\") pod \"4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1\" (UID: \"4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1\") " Jul 09 04:36:47.412374 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.412217 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1-openshift-service-ca\") pod \"4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1\" (UID: \"4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1\") " Jul 09 04:36:47.412649 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.412619 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1" (UID: "4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:36:47.414639 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.414606 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1-kube-api-access-7lsdp" (OuterVolumeSpecName: "kube-api-access-7lsdp") pod "4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1" (UID: "4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1"). InnerVolumeSpecName "kube-api-access-7lsdp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:36:47.414793 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.414751 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1" (UID: "4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:36:47.513266 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.513178 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lsdp\" (UniqueName: \"kubernetes.io/projected/4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1-kube-api-access-7lsdp\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:36:47.513266 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.513209 2534 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1-maas-api-tls\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:36:47.513266 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.513223 2534 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1-openshift-service-ca\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:36:47.675821 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.675786 2534 generic.go:356] "Generic (PLEG): container finished" podID="439b293c-fc3a-4ab8-a6c9-16e57815cf49" containerID="4648dfa2a351449ad972df1982b89bb756cd0a95f23c5493aae211945623f5e1" exitCode=0 Jul 09 04:36:47.675984 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.675852 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" Jul 09 04:36:47.675984 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.675877 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" event={"ID":"439b293c-fc3a-4ab8-a6c9-16e57815cf49","Type":"ContainerDied","Data":"4648dfa2a351449ad972df1982b89bb756cd0a95f23c5493aae211945623f5e1"} Jul 09 04:36:47.675984 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.675930 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89" event={"ID":"439b293c-fc3a-4ab8-a6c9-16e57815cf49","Type":"ContainerDied","Data":"5bf14438ed116307a507fe4a3ebd29d6b93d5d1c6b6a062c83d8c36fa274c6fa"} Jul 09 04:36:47.675984 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.675952 2534 scope.go:117] "RemoveContainer" containerID="4648dfa2a351449ad972df1982b89bb756cd0a95f23c5493aae211945623f5e1" Jul 09 04:36:47.677306 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.677259 2534 generic.go:356] "Generic (PLEG): container finished" podID="4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1" containerID="cdf00bec38eeb83f44f12ecaed7ca20bf3856300d25d8256c8724bad4025b602" exitCode=0 Jul 09 04:36:47.677422 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.677310 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv" event={"ID":"4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1","Type":"ContainerDied","Data":"cdf00bec38eeb83f44f12ecaed7ca20bf3856300d25d8256c8724bad4025b602"} Jul 09 04:36:47.677422 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.677340 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv" Jul 09 04:36:47.677422 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.677355 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv" event={"ID":"4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1","Type":"ContainerDied","Data":"2b89fc4c83cb090a153a38a04bb158c7e87e85bc59176a8dac714bfefba026d7"} Jul 09 04:36:47.686644 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.686622 2534 scope.go:117] "RemoveContainer" containerID="ede17f42e542395577fa650399636c03940b6f513bed7166ffc2fae8735e9e7c" Jul 09 04:36:47.695394 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.695372 2534 scope.go:117] "RemoveContainer" containerID="4648dfa2a351449ad972df1982b89bb756cd0a95f23c5493aae211945623f5e1" Jul 09 04:36:47.695637 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:36:47.695618 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4648dfa2a351449ad972df1982b89bb756cd0a95f23c5493aae211945623f5e1\": container with ID starting with 4648dfa2a351449ad972df1982b89bb756cd0a95f23c5493aae211945623f5e1 not found: ID does not exist" containerID="4648dfa2a351449ad972df1982b89bb756cd0a95f23c5493aae211945623f5e1" Jul 09 04:36:47.695707 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.695644 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4648dfa2a351449ad972df1982b89bb756cd0a95f23c5493aae211945623f5e1"} err="failed to get container status \"4648dfa2a351449ad972df1982b89bb756cd0a95f23c5493aae211945623f5e1\": rpc error: code = NotFound desc = could not find container \"4648dfa2a351449ad972df1982b89bb756cd0a95f23c5493aae211945623f5e1\": container with ID starting with 4648dfa2a351449ad972df1982b89bb756cd0a95f23c5493aae211945623f5e1 not found: ID does not exist" Jul 09 04:36:47.695707 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.695664 2534 scope.go:117] "RemoveContainer" containerID="ede17f42e542395577fa650399636c03940b6f513bed7166ffc2fae8735e9e7c" Jul 09 04:36:47.695887 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:36:47.695870 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ede17f42e542395577fa650399636c03940b6f513bed7166ffc2fae8735e9e7c\": container with ID starting with ede17f42e542395577fa650399636c03940b6f513bed7166ffc2fae8735e9e7c not found: ID does not exist" containerID="ede17f42e542395577fa650399636c03940b6f513bed7166ffc2fae8735e9e7c" Jul 09 04:36:47.695926 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.695895 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ede17f42e542395577fa650399636c03940b6f513bed7166ffc2fae8735e9e7c"} err="failed to get container status \"ede17f42e542395577fa650399636c03940b6f513bed7166ffc2fae8735e9e7c\": rpc error: code = NotFound desc = could not find container \"ede17f42e542395577fa650399636c03940b6f513bed7166ffc2fae8735e9e7c\": container with ID starting with ede17f42e542395577fa650399636c03940b6f513bed7166ffc2fae8735e9e7c not found: ID does not exist" Jul 09 04:36:47.695926 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.695910 2534 scope.go:117] "RemoveContainer" containerID="cdf00bec38eeb83f44f12ecaed7ca20bf3856300d25d8256c8724bad4025b602" Jul 09 04:36:47.702239 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.702214 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89"] Jul 09 04:36:47.704108 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.704089 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-51ad45e2-openshift-default-648c6f8465-nxp89"] Jul 09 04:36:47.705419 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.705400 2534 scope.go:117] "RemoveContainer" containerID="cdf00bec38eeb83f44f12ecaed7ca20bf3856300d25d8256c8724bad4025b602" Jul 09 04:36:47.705718 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:36:47.705698 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdf00bec38eeb83f44f12ecaed7ca20bf3856300d25d8256c8724bad4025b602\": container with ID starting with cdf00bec38eeb83f44f12ecaed7ca20bf3856300d25d8256c8724bad4025b602 not found: ID does not exist" containerID="cdf00bec38eeb83f44f12ecaed7ca20bf3856300d25d8256c8724bad4025b602" Jul 09 04:36:47.705774 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.705726 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdf00bec38eeb83f44f12ecaed7ca20bf3856300d25d8256c8724bad4025b602"} err="failed to get container status \"cdf00bec38eeb83f44f12ecaed7ca20bf3856300d25d8256c8724bad4025b602\": rpc error: code = NotFound desc = could not find container \"cdf00bec38eeb83f44f12ecaed7ca20bf3856300d25d8256c8724bad4025b602\": container with ID starting with cdf00bec38eeb83f44f12ecaed7ca20bf3856300d25d8256c8724bad4025b602 not found: ID does not exist" Jul 09 04:36:47.714347 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.714316 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv"] Jul 09 04:36:47.718325 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:47.718305 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-51ad45e2-7cd48dcb4d-9vblv"] Jul 09 04:36:48.250258 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:48.250177 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="439b293c-fc3a-4ab8-a6c9-16e57815cf49" path="/var/lib/kubelet/pods/439b293c-fc3a-4ab8-a6c9-16e57815cf49/volumes" Jul 09 04:36:48.250652 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:48.250612 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1" path="/var/lib/kubelet/pods/4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1/volumes" Jul 09 04:36:50.335610 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:50.335586 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8dnq_633a3ae5-3d78-4217-901a-6d0d06442cf4/console-operator/1.log" Jul 09 04:36:50.340829 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:50.340803 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8dnq_633a3ae5-3d78-4217-901a-6d0d06442cf4/console-operator/1.log" Jul 09 04:36:53.463178 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.463131 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c"] Jul 09 04:36:53.463906 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.463847 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="439b293c-fc3a-4ab8-a6c9-16e57815cf49" containerName="istio-proxy" Jul 09 04:36:53.464079 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.464059 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="439b293c-fc3a-4ab8-a6c9-16e57815cf49" containerName="istio-proxy" Jul 09 04:36:53.464187 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.464104 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1" containerName="maas-api" Jul 09 04:36:53.464187 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.464114 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1" containerName="maas-api" Jul 09 04:36:53.464187 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.464134 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="439b293c-fc3a-4ab8-a6c9-16e57815cf49" containerName="istio-proxy" Jul 09 04:36:53.464187 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.464142 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="439b293c-fc3a-4ab8-a6c9-16e57815cf49" containerName="istio-proxy" Jul 09 04:36:53.464355 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.464262 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="439b293c-fc3a-4ab8-a6c9-16e57815cf49" containerName="istio-proxy" Jul 09 04:36:53.464355 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.464274 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="439b293c-fc3a-4ab8-a6c9-16e57815cf49" containerName="istio-proxy" Jul 09 04:36:53.464355 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.464312 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bb1c3d3-824a-4ddb-a932-c1c9cd2206e1" containerName="maas-api" Jul 09 04:36:53.467772 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.467754 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.470407 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.470385 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-82cecee7-openshift-default-dockercfg-mxh8g\"" Jul 09 04:36:53.480542 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.480516 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c"] Jul 09 04:36:53.571811 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.571780 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-istio-data\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.572007 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.571816 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f908907-acc0-4d64-83f3-598671a904a9-istiod-ca-cert\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.572007 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.571845 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-istio-envoy\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.572007 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.571896 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f908907-acc0-4d64-83f3-598671a904a9-istio-token\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.572166 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.572012 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ml9dd\" (UniqueName: \"kubernetes.io/projected/5f908907-acc0-4d64-83f3-598671a904a9-kube-api-access-ml9dd\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.572166 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.572058 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-workload-certs\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.572166 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.572086 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-workload-socket\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.572166 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.572130 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-credential-socket\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.572166 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.572163 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f908907-acc0-4d64-83f3-598671a904a9-istio-podinfo\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.672892 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.672855 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-istio-data\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.672892 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.672894 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f908907-acc0-4d64-83f3-598671a904a9-istiod-ca-cert\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.673142 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.672917 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-istio-envoy\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.673142 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.672936 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f908907-acc0-4d64-83f3-598671a904a9-istio-token\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.673142 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.672965 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ml9dd\" (UniqueName: \"kubernetes.io/projected/5f908907-acc0-4d64-83f3-598671a904a9-kube-api-access-ml9dd\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.673142 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.673005 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-workload-certs\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.673142 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.673032 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-workload-socket\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.673142 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.673056 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-credential-socket\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.673142 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.673091 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f908907-acc0-4d64-83f3-598671a904a9-istio-podinfo\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.673515 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.673407 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-istio-data\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.673582 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.673509 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-workload-certs\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.673582 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.673553 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-credential-socket\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.673682 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.673616 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-workload-socket\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.673743 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.673678 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f908907-acc0-4d64-83f3-598671a904a9-istiod-ca-cert\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.675501 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.675480 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-istio-envoy\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.675812 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.675795 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f908907-acc0-4d64-83f3-598671a904a9-istio-podinfo\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.681000 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.680972 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ml9dd\" (UniqueName: \"kubernetes.io/projected/5f908907-acc0-4d64-83f3-598671a904a9-kube-api-access-ml9dd\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.681095 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.680999 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f908907-acc0-4d64-83f3-598671a904a9-istio-token\") pod \"e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.780918 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.780836 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:53.945166 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.945137 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c"] Jul 09 04:36:53.947087 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:36:53.947056 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f908907_acc0_4d64_83f3_598671a904a9.slice/crio-32ab89da012dfca5fa9766296bcf4a7667094c62474abafbd8822baac6a8e6f0 WatchSource:0}: Error finding container 32ab89da012dfca5fa9766296bcf4a7667094c62474abafbd8822baac6a8e6f0: Status 404 returned error can't find the container with id 32ab89da012dfca5fa9766296bcf4a7667094c62474abafbd8822baac6a8e6f0 Jul 09 04:36:53.949671 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.949636 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:36:53.949756 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.949702 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:36:53.949810 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:53.949758 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:36:54.721509 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:54.721475 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" event={"ID":"5f908907-acc0-4d64-83f3-598671a904a9","Type":"ContainerStarted","Data":"8704491c1c49b6abbb86cd32da6b74fcf7db75e7bcf0040e26dc9cf58c74627b"} Jul 09 04:36:54.721889 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:54.721530 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" event={"ID":"5f908907-acc0-4d64-83f3-598671a904a9","Type":"ContainerStarted","Data":"32ab89da012dfca5fa9766296bcf4a7667094c62474abafbd8822baac6a8e6f0"} Jul 09 04:36:54.743114 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:54.743056 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" podStartSLOduration=1.743034025 podStartE2EDuration="1.743034025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:36:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:36:54.742399992 +0000 UTC m=+1804.966018480" watchObservedRunningTime="2026-07-09 04:36:54.743034025 +0000 UTC m=+1804.966652515" Jul 09 04:36:54.780983 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:54.780952 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:55.785910 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:55.785882 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:56.732819 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:56.732789 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:36:56.733804 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:36:56.733783 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:39:58.102331 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.102243 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c"] Jul 09 04:39:58.102836 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.102472 2534 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" podUID="5f908907-acc0-4d64-83f3-598671a904a9" containerName="istio-proxy" containerID="cri-o://8704491c1c49b6abbb86cd32da6b74fcf7db75e7bcf0040e26dc9cf58c74627b" gracePeriod=30 Jul 09 04:39:58.458390 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.458320 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl"] Jul 09 04:39:58.462006 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.461988 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.464484 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.464269 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-c5d9fea6-openshift-default-dockercfg-mxzfs\"" Jul 09 04:39:58.474179 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.474157 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl"] Jul 09 04:39:58.510464 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.510432 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istio-envoy\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.510618 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.510484 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-workload-certs\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.510618 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.510515 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istio-podinfo\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.510618 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.510545 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-credential-socket\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.510618 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.510567 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istio-token\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.510757 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.510620 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62crz\" (UniqueName: \"kubernetes.io/projected/a3659f35-ee7e-4158-8a49-cbfc454feeb9-kube-api-access-62crz\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.510757 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.510656 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-workload-socket\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.510757 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.510681 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istio-data\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.510757 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.510700 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istiod-ca-cert\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.611768 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.611735 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-workload-socket\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.611940 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.611778 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istio-data\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.611940 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.611795 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istiod-ca-cert\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.611940 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.611832 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istio-envoy\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.611940 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.611869 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-workload-certs\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.611940 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.611889 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istio-podinfo\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.611940 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.611924 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-credential-socket\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.612275 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.611951 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istio-token\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.612275 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.611978 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62crz\" (UniqueName: \"kubernetes.io/projected/a3659f35-ee7e-4158-8a49-cbfc454feeb9-kube-api-access-62crz\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.612275 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.612216 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istio-data\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.612452 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.612314 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-credential-socket\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.612452 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.612400 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-workload-certs\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.612542 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.612515 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-workload-socket\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.612862 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.612838 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istiod-ca-cert\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.614357 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.614334 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istio-envoy\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.614592 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.614568 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istio-podinfo\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.623160 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.623135 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62crz\" (UniqueName: \"kubernetes.io/projected/a3659f35-ee7e-4158-8a49-cbfc454feeb9-kube-api-access-62crz\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.623697 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.623677 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istio-token\") pod \"e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.774730 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.774653 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:58.914360 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.914332 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl"] Jul 09 04:39:58.916333 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:39:58.916311 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3659f35_ee7e_4158_8a49_cbfc454feeb9.slice/crio-ed61adc1f215dce5f7266f5a1bb5c94106e67eddf60fa6102eb546e2b7fa7efb WatchSource:0}: Error finding container ed61adc1f215dce5f7266f5a1bb5c94106e67eddf60fa6102eb546e2b7fa7efb: Status 404 returned error can't find the container with id ed61adc1f215dce5f7266f5a1bb5c94106e67eddf60fa6102eb546e2b7fa7efb Jul 09 04:39:58.918462 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.918428 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:39:58.918554 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.918493 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:39:58.918554 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:58.918522 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:39:59.693514 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:59.693479 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" event={"ID":"a3659f35-ee7e-4158-8a49-cbfc454feeb9","Type":"ContainerStarted","Data":"98859ff10b942fa441236590fdff92de4756958d3465e11ac586fe9e9d78584c"} Jul 09 04:39:59.693886 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:59.693561 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" event={"ID":"a3659f35-ee7e-4158-8a49-cbfc454feeb9","Type":"ContainerStarted","Data":"ed61adc1f215dce5f7266f5a1bb5c94106e67eddf60fa6102eb546e2b7fa7efb"} Jul 09 04:39:59.712926 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:59.712885 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" podStartSLOduration=1.712864671 podStartE2EDuration="1.712864671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:39:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:39:59.711097513 +0000 UTC m=+1989.934716000" watchObservedRunningTime="2026-07-09 04:39:59.712864671 +0000 UTC m=+1989.936483158" Jul 09 04:39:59.775578 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:59.775550 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:39:59.780345 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:39:59.780323 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:40:00.701054 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:00.701022 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:40:00.702136 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:00.702118 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:40:03.345532 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.345509 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:40:03.464815 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.464784 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-credential-socket\") pod \"5f908907-acc0-4d64-83f3-598671a904a9\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " Jul 09 04:40:03.464976 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.464832 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f908907-acc0-4d64-83f3-598671a904a9-istiod-ca-cert\") pod \"5f908907-acc0-4d64-83f3-598671a904a9\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " Jul 09 04:40:03.464976 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.464873 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ml9dd\" (UniqueName: \"kubernetes.io/projected/5f908907-acc0-4d64-83f3-598671a904a9-kube-api-access-ml9dd\") pod \"5f908907-acc0-4d64-83f3-598671a904a9\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " Jul 09 04:40:03.464976 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.464923 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-workload-certs\") pod \"5f908907-acc0-4d64-83f3-598671a904a9\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " Jul 09 04:40:03.464976 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.464962 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f908907-acc0-4d64-83f3-598671a904a9-istio-podinfo\") pod \"5f908907-acc0-4d64-83f3-598671a904a9\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " Jul 09 04:40:03.465179 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.464990 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-istio-data\") pod \"5f908907-acc0-4d64-83f3-598671a904a9\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " Jul 09 04:40:03.465179 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.465014 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f908907-acc0-4d64-83f3-598671a904a9-istio-token\") pod \"5f908907-acc0-4d64-83f3-598671a904a9\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " Jul 09 04:40:03.465179 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.465050 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-workload-socket\") pod \"5f908907-acc0-4d64-83f3-598671a904a9\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " Jul 09 04:40:03.465377 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.465311 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-istio-envoy\") pod \"5f908907-acc0-4d64-83f3-598671a904a9\" (UID: \"5f908907-acc0-4d64-83f3-598671a904a9\") " Jul 09 04:40:03.465377 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.465044 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5f908907-acc0-4d64-83f3-598671a904a9" (UID: "5f908907-acc0-4d64-83f3-598671a904a9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:40:03.465377 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.465334 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f908907-acc0-4d64-83f3-598671a904a9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5f908907-acc0-4d64-83f3-598671a904a9" (UID: "5f908907-acc0-4d64-83f3-598671a904a9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:40:03.465377 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.465307 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5f908907-acc0-4d64-83f3-598671a904a9" (UID: "5f908907-acc0-4d64-83f3-598671a904a9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:40:03.465377 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.465374 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-istio-data" (OuterVolumeSpecName: "istio-data") pod "5f908907-acc0-4d64-83f3-598671a904a9" (UID: "5f908907-acc0-4d64-83f3-598671a904a9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:40:03.465674 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.465388 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5f908907-acc0-4d64-83f3-598671a904a9" (UID: "5f908907-acc0-4d64-83f3-598671a904a9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:40:03.466026 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.466003 2534 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-workload-certs\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:40:03.466099 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.466030 2534 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-istio-data\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:40:03.466099 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.466044 2534 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-workload-socket\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:40:03.466099 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.466059 2534 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-credential-socket\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:40:03.466099 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.466075 2534 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f908907-acc0-4d64-83f3-598671a904a9-istiod-ca-cert\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:40:03.467176 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.467155 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f908907-acc0-4d64-83f3-598671a904a9-kube-api-access-ml9dd" (OuterVolumeSpecName: "kube-api-access-ml9dd") pod "5f908907-acc0-4d64-83f3-598671a904a9" (UID: "5f908907-acc0-4d64-83f3-598671a904a9"). InnerVolumeSpecName "kube-api-access-ml9dd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:40:03.467305 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.467186 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f908907-acc0-4d64-83f3-598671a904a9-istio-token" (OuterVolumeSpecName: "istio-token") pod "5f908907-acc0-4d64-83f3-598671a904a9" (UID: "5f908907-acc0-4d64-83f3-598671a904a9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:40:03.467365 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.467305 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5f908907-acc0-4d64-83f3-598671a904a9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5f908907-acc0-4d64-83f3-598671a904a9" (UID: "5f908907-acc0-4d64-83f3-598671a904a9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 04:40:03.467365 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.467357 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5f908907-acc0-4d64-83f3-598671a904a9" (UID: "5f908907-acc0-4d64-83f3-598671a904a9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:40:03.566741 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.566708 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ml9dd\" (UniqueName: \"kubernetes.io/projected/5f908907-acc0-4d64-83f3-598671a904a9-kube-api-access-ml9dd\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:40:03.566741 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.566736 2534 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f908907-acc0-4d64-83f3-598671a904a9-istio-podinfo\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:40:03.566741 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.566745 2534 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f908907-acc0-4d64-83f3-598671a904a9-istio-token\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:40:03.566995 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.566754 2534 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f908907-acc0-4d64-83f3-598671a904a9-istio-envoy\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:40:03.716869 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.716787 2534 generic.go:356] "Generic (PLEG): container finished" podID="5f908907-acc0-4d64-83f3-598671a904a9" containerID="8704491c1c49b6abbb86cd32da6b74fcf7db75e7bcf0040e26dc9cf58c74627b" exitCode=0 Jul 09 04:40:03.716869 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.716839 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" Jul 09 04:40:03.717103 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.716877 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" event={"ID":"5f908907-acc0-4d64-83f3-598671a904a9","Type":"ContainerDied","Data":"8704491c1c49b6abbb86cd32da6b74fcf7db75e7bcf0040e26dc9cf58c74627b"} Jul 09 04:40:03.717103 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.716926 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c" event={"ID":"5f908907-acc0-4d64-83f3-598671a904a9","Type":"ContainerDied","Data":"32ab89da012dfca5fa9766296bcf4a7667094c62474abafbd8822baac6a8e6f0"} Jul 09 04:40:03.717103 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.716949 2534 scope.go:117] "RemoveContainer" containerID="8704491c1c49b6abbb86cd32da6b74fcf7db75e7bcf0040e26dc9cf58c74627b" Jul 09 04:40:03.726637 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.726615 2534 scope.go:117] "RemoveContainer" containerID="8704491c1c49b6abbb86cd32da6b74fcf7db75e7bcf0040e26dc9cf58c74627b" Jul 09 04:40:03.726931 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:40:03.726912 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8704491c1c49b6abbb86cd32da6b74fcf7db75e7bcf0040e26dc9cf58c74627b\": container with ID starting with 8704491c1c49b6abbb86cd32da6b74fcf7db75e7bcf0040e26dc9cf58c74627b not found: ID does not exist" containerID="8704491c1c49b6abbb86cd32da6b74fcf7db75e7bcf0040e26dc9cf58c74627b" Jul 09 04:40:03.727014 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.726944 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8704491c1c49b6abbb86cd32da6b74fcf7db75e7bcf0040e26dc9cf58c74627b"} err="failed to get container status \"8704491c1c49b6abbb86cd32da6b74fcf7db75e7bcf0040e26dc9cf58c74627b\": rpc error: code = NotFound desc = could not find container \"8704491c1c49b6abbb86cd32da6b74fcf7db75e7bcf0040e26dc9cf58c74627b\": container with ID starting with 8704491c1c49b6abbb86cd32da6b74fcf7db75e7bcf0040e26dc9cf58c74627b not found: ID does not exist" Jul 09 04:40:03.737222 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.737198 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c"] Jul 09 04:40:03.740729 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:03.740709 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-82cecee7-openshift-default-86d4555947-dsx4c"] Jul 09 04:40:04.250980 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:40:04.250944 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f908907-acc0-4d64-83f3-598671a904a9" path="/var/lib/kubelet/pods/5f908907-acc0-4d64-83f3-598671a904a9/volumes" Jul 09 04:41:50.371623 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:41:50.371594 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8dnq_633a3ae5-3d78-4217-901a-6d0d06442cf4/console-operator/1.log" Jul 09 04:41:50.379238 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:41:50.379212 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8dnq_633a3ae5-3d78-4217-901a-6d0d06442cf4/console-operator/1.log" Jul 09 04:42:01.652603 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:01.652559 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl"] Jul 09 04:42:01.653062 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:01.652799 2534 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" podUID="a3659f35-ee7e-4158-8a49-cbfc454feeb9" containerName="istio-proxy" containerID="cri-o://98859ff10b942fa441236590fdff92de4756958d3465e11ac586fe9e9d78584c" gracePeriod=30 Jul 09 04:42:06.905489 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:06.905458 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:42:07.029376 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.029348 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istio-data\") pod \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " Jul 09 04:42:07.029537 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.029386 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istio-envoy\") pod \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " Jul 09 04:42:07.029537 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.029437 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istio-podinfo\") pod \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " Jul 09 04:42:07.029537 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.029456 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-credential-socket\") pod \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " Jul 09 04:42:07.029537 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.029477 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-workload-certs\") pod \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " Jul 09 04:42:07.029537 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.029507 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istiod-ca-cert\") pod \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " Jul 09 04:42:07.029841 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.029547 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-workload-socket\") pod \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " Jul 09 04:42:07.029841 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.029589 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istio-token\") pod \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " Jul 09 04:42:07.029841 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.029621 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62crz\" (UniqueName: \"kubernetes.io/projected/a3659f35-ee7e-4158-8a49-cbfc454feeb9-kube-api-access-62crz\") pod \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\" (UID: \"a3659f35-ee7e-4158-8a49-cbfc454feeb9\") " Jul 09 04:42:07.029841 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.029646 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istio-data" (OuterVolumeSpecName: "istio-data") pod "a3659f35-ee7e-4158-8a49-cbfc454feeb9" (UID: "a3659f35-ee7e-4158-8a49-cbfc454feeb9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:42:07.029841 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.029813 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a3659f35-ee7e-4158-8a49-cbfc454feeb9" (UID: "a3659f35-ee7e-4158-8a49-cbfc454feeb9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:42:07.030055 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.029838 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a3659f35-ee7e-4158-8a49-cbfc454feeb9" (UID: "a3659f35-ee7e-4158-8a49-cbfc454feeb9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:42:07.030055 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.029895 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a3659f35-ee7e-4158-8a49-cbfc454feeb9" (UID: "a3659f35-ee7e-4158-8a49-cbfc454feeb9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:42:07.030055 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.029987 2534 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-workload-socket\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:42:07.030055 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.030007 2534 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istio-data\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:42:07.030055 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.030021 2534 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-credential-socket\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:42:07.030055 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.030034 2534 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-workload-certs\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:42:07.030271 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.030249 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a3659f35-ee7e-4158-8a49-cbfc454feeb9" (UID: "a3659f35-ee7e-4158-8a49-cbfc454feeb9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:42:07.031998 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.031967 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a3659f35-ee7e-4158-8a49-cbfc454feeb9" (UID: "a3659f35-ee7e-4158-8a49-cbfc454feeb9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:42:07.032138 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.032105 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a3659f35-ee7e-4158-8a49-cbfc454feeb9" (UID: "a3659f35-ee7e-4158-8a49-cbfc454feeb9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 04:42:07.032361 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.032328 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3659f35-ee7e-4158-8a49-cbfc454feeb9-kube-api-access-62crz" (OuterVolumeSpecName: "kube-api-access-62crz") pod "a3659f35-ee7e-4158-8a49-cbfc454feeb9" (UID: "a3659f35-ee7e-4158-8a49-cbfc454feeb9"). InnerVolumeSpecName "kube-api-access-62crz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:42:07.032460 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.032384 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istio-token" (OuterVolumeSpecName: "istio-token") pod "a3659f35-ee7e-4158-8a49-cbfc454feeb9" (UID: "a3659f35-ee7e-4158-8a49-cbfc454feeb9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:42:07.131230 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.131191 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62crz\" (UniqueName: \"kubernetes.io/projected/a3659f35-ee7e-4158-8a49-cbfc454feeb9-kube-api-access-62crz\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:42:07.131230 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.131225 2534 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istio-envoy\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:42:07.131230 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.131238 2534 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istio-podinfo\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:42:07.131486 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.131248 2534 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istiod-ca-cert\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:42:07.131486 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.131261 2534 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3659f35-ee7e-4158-8a49-cbfc454feeb9-istio-token\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:42:07.345820 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.345790 2534 generic.go:356] "Generic (PLEG): container finished" podID="a3659f35-ee7e-4158-8a49-cbfc454feeb9" containerID="98859ff10b942fa441236590fdff92de4756958d3465e11ac586fe9e9d78584c" exitCode=0 Jul 09 04:42:07.346020 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.345840 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" Jul 09 04:42:07.346020 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.345874 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" event={"ID":"a3659f35-ee7e-4158-8a49-cbfc454feeb9","Type":"ContainerDied","Data":"98859ff10b942fa441236590fdff92de4756958d3465e11ac586fe9e9d78584c"} Jul 09 04:42:07.346020 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.345928 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl" event={"ID":"a3659f35-ee7e-4158-8a49-cbfc454feeb9","Type":"ContainerDied","Data":"ed61adc1f215dce5f7266f5a1bb5c94106e67eddf60fa6102eb546e2b7fa7efb"} Jul 09 04:42:07.346020 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.345949 2534 scope.go:117] "RemoveContainer" containerID="98859ff10b942fa441236590fdff92de4756958d3465e11ac586fe9e9d78584c" Jul 09 04:42:07.355887 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.355869 2534 scope.go:117] "RemoveContainer" containerID="98859ff10b942fa441236590fdff92de4756958d3465e11ac586fe9e9d78584c" Jul 09 04:42:07.356143 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:42:07.356125 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98859ff10b942fa441236590fdff92de4756958d3465e11ac586fe9e9d78584c\": container with ID starting with 98859ff10b942fa441236590fdff92de4756958d3465e11ac586fe9e9d78584c not found: ID does not exist" containerID="98859ff10b942fa441236590fdff92de4756958d3465e11ac586fe9e9d78584c" Jul 09 04:42:07.356199 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.356150 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98859ff10b942fa441236590fdff92de4756958d3465e11ac586fe9e9d78584c"} err="failed to get container status \"98859ff10b942fa441236590fdff92de4756958d3465e11ac586fe9e9d78584c\": rpc error: code = NotFound desc = could not find container \"98859ff10b942fa441236590fdff92de4756958d3465e11ac586fe9e9d78584c\": container with ID starting with 98859ff10b942fa441236590fdff92de4756958d3465e11ac586fe9e9d78584c not found: ID does not exist" Jul 09 04:42:07.366994 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.366969 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl"] Jul 09 04:42:07.372696 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:07.372673 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-c5d9fea6-openshift-default-6866fd8b67-jbkdl"] Jul 09 04:42:08.250086 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:42:08.250051 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3659f35-ee7e-4158-8a49-cbfc454feeb9" path="/var/lib/kubelet/pods/a3659f35-ee7e-4158-8a49-cbfc454feeb9/volumes" Jul 09 04:43:36.053059 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.053026 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz"] Jul 09 04:43:36.053716 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.053695 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3659f35-ee7e-4158-8a49-cbfc454feeb9" containerName="istio-proxy" Jul 09 04:43:36.053797 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.053718 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3659f35-ee7e-4158-8a49-cbfc454feeb9" containerName="istio-proxy" Jul 09 04:43:36.053797 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.053739 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f908907-acc0-4d64-83f3-598671a904a9" containerName="istio-proxy" Jul 09 04:43:36.053797 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.053747 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f908907-acc0-4d64-83f3-598671a904a9" containerName="istio-proxy" Jul 09 04:43:36.053951 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.053859 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f908907-acc0-4d64-83f3-598671a904a9" containerName="istio-proxy" Jul 09 04:43:36.053951 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.053877 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3659f35-ee7e-4158-8a49-cbfc454feeb9" containerName="istio-proxy" Jul 09 04:43:36.057041 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.057019 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.059314 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.059273 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-d1630ae3-openshift-default-dockercfg-276bx\"" Jul 09 04:43:36.071397 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.071365 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz"] Jul 09 04:43:36.170477 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.170445 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj5v2\" (UniqueName: \"kubernetes.io/projected/a5f3e447-0444-419a-a3e8-840655333cd2-kube-api-access-dj5v2\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.170656 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.170484 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5f3e447-0444-419a-a3e8-840655333cd2-istio-podinfo\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.170656 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.170597 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5f3e447-0444-419a-a3e8-840655333cd2-istio-token\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.170656 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.170631 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5f3e447-0444-419a-a3e8-840655333cd2-istiod-ca-cert\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.170768 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.170661 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-workload-certs\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.170768 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.170696 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-credential-socket\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.170768 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.170736 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-workload-socket\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.170861 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.170819 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-istio-envoy\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.170861 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.170847 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-istio-data\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.271555 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.271518 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-istio-envoy\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.271555 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.271555 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-istio-data\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.271790 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.271577 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj5v2\" (UniqueName: \"kubernetes.io/projected/a5f3e447-0444-419a-a3e8-840655333cd2-kube-api-access-dj5v2\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.271790 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.271605 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5f3e447-0444-419a-a3e8-840655333cd2-istio-podinfo\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.271790 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.271686 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5f3e447-0444-419a-a3e8-840655333cd2-istio-token\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.271790 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.271715 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5f3e447-0444-419a-a3e8-840655333cd2-istiod-ca-cert\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.271790 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.271739 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-workload-certs\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.271790 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.271763 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-credential-socket\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.272107 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.271858 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-workload-socket\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.272107 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.272062 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-istio-data\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.272329 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.272265 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-credential-socket\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.272420 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.272276 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-workload-certs\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.272568 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.272542 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-workload-socket\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.272568 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.272555 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5f3e447-0444-419a-a3e8-840655333cd2-istiod-ca-cert\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.274049 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.274029 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-istio-envoy\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.274262 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.274243 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5f3e447-0444-419a-a3e8-840655333cd2-istio-podinfo\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.280087 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.280061 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5f3e447-0444-419a-a3e8-840655333cd2-istio-token\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.280535 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.280518 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj5v2\" (UniqueName: \"kubernetes.io/projected/a5f3e447-0444-419a-a3e8-840655333cd2-kube-api-access-dj5v2\") pod \"e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.377514 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.377420 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:36.519877 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.519855 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz"] Jul 09 04:43:36.522230 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:43:36.522203 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5f3e447_0444_419a_a3e8_840655333cd2.slice/crio-b6ed550e0d82c2c89776d7e983cb6a687401e3cad145df205e6cb7afc4e25a7c WatchSource:0}: Error finding container b6ed550e0d82c2c89776d7e983cb6a687401e3cad145df205e6cb7afc4e25a7c: Status 404 returned error can't find the container with id b6ed550e0d82c2c89776d7e983cb6a687401e3cad145df205e6cb7afc4e25a7c Jul 09 04:43:36.524579 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.524539 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:43:36.524647 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.524632 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:43:36.524691 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.524676 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:43:36.805782 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.805740 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" event={"ID":"a5f3e447-0444-419a-a3e8-840655333cd2","Type":"ContainerStarted","Data":"b49c653f592464588fa3b732a3bff82964e6dc61a2b59c44be0ecbefc504165c"} Jul 09 04:43:36.805973 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.805805 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" event={"ID":"a5f3e447-0444-419a-a3e8-840655333cd2","Type":"ContainerStarted","Data":"b6ed550e0d82c2c89776d7e983cb6a687401e3cad145df205e6cb7afc4e25a7c"} Jul 09 04:43:36.824453 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:36.824405 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" podStartSLOduration=0.824388293 podStartE2EDuration="824.388293ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:43:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:43:36.821675936 +0000 UTC m=+2207.045294423" watchObservedRunningTime="2026-07-09 04:43:36.824388293 +0000 UTC m=+2207.048006781" Jul 09 04:43:37.377901 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:37.377869 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:37.383138 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:37.383111 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:37.811293 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:37.811253 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:43:37.812121 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:43:37.812102 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:45:00.134085 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:45:00.134054 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726205-fqkxl"] Jul 09 04:45:00.137575 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:45:00.137559 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726205-fqkxl" Jul 09 04:45:00.139834 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:45:00.139811 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7qtgv\"" Jul 09 04:45:00.144180 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:45:00.144157 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726205-fqkxl"] Jul 09 04:45:00.148784 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:45:00.148764 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726205-fqkxl" Jul 09 04:45:00.283697 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:45:00.283667 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726205-fqkxl"] Jul 09 04:45:00.286523 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:45:00.286488 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaec72baa_af39_4268_921f_080fad640ca6.slice/crio-ed82e570d53ef0c532652b85c94784fdb129e0d7b68b7a799bc5fedf74fab5ad WatchSource:0}: Error finding container ed82e570d53ef0c532652b85c94784fdb129e0d7b68b7a799bc5fedf74fab5ad: Status 404 returned error can't find the container with id ed82e570d53ef0c532652b85c94784fdb129e0d7b68b7a799bc5fedf74fab5ad Jul 09 04:45:01.240063 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:45:01.239971 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726205-fqkxl" event={"ID":"aec72baa-af39-4268-921f-080fad640ca6","Type":"ContainerStarted","Data":"b7588ba87288277d30cc36758f9f04a8f4cc536df2c5503ec1f179f38bceb60a"} Jul 09 04:45:01.240063 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:45:01.240025 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726205-fqkxl" event={"ID":"aec72baa-af39-4268-921f-080fad640ca6","Type":"ContainerStarted","Data":"ed82e570d53ef0c532652b85c94784fdb129e0d7b68b7a799bc5fedf74fab5ad"} Jul 09 04:45:02.253817 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:45:02.253785 2534 generic.go:356] "Generic (PLEG): container finished" podID="aec72baa-af39-4268-921f-080fad640ca6" containerID="b7588ba87288277d30cc36758f9f04a8f4cc536df2c5503ec1f179f38bceb60a" exitCode=0 Jul 09 04:45:02.256319 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:45:02.256267 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726205-fqkxl" event={"ID":"aec72baa-af39-4268-921f-080fad640ca6","Type":"ContainerDied","Data":"b7588ba87288277d30cc36758f9f04a8f4cc536df2c5503ec1f179f38bceb60a"} Jul 09 04:45:02.389980 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:45:02.389952 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726205-fqkxl" Jul 09 04:45:03.260572 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:45:03.260540 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726205-fqkxl" event={"ID":"aec72baa-af39-4268-921f-080fad640ca6","Type":"ContainerDied","Data":"ed82e570d53ef0c532652b85c94784fdb129e0d7b68b7a799bc5fedf74fab5ad"} Jul 09 04:45:03.260943 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:45:03.260578 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726205-fqkxl" Jul 09 04:45:03.260943 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:45:03.260584 2534 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed82e570d53ef0c532652b85c94784fdb129e0d7b68b7a799bc5fedf74fab5ad" Jul 09 04:46:43.058725 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:43.058690 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz"] Jul 09 04:46:43.059253 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:43.058946 2534 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" podUID="a5f3e447-0444-419a-a3e8-840655333cd2" containerName="istio-proxy" containerID="cri-o://b49c653f592464588fa3b732a3bff82964e6dc61a2b59c44be0ecbefc504165c" gracePeriod=30 Jul 09 04:46:48.317467 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.317444 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:46:48.365217 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.365138 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5f3e447-0444-419a-a3e8-840655333cd2-istio-token\") pod \"a5f3e447-0444-419a-a3e8-840655333cd2\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " Jul 09 04:46:48.365217 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.365193 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-workload-certs\") pod \"a5f3e447-0444-419a-a3e8-840655333cd2\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " Jul 09 04:46:48.365472 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.365310 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dj5v2\" (UniqueName: \"kubernetes.io/projected/a5f3e447-0444-419a-a3e8-840655333cd2-kube-api-access-dj5v2\") pod \"a5f3e447-0444-419a-a3e8-840655333cd2\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " Jul 09 04:46:48.365472 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.365353 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5f3e447-0444-419a-a3e8-840655333cd2-istiod-ca-cert\") pod \"a5f3e447-0444-419a-a3e8-840655333cd2\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " Jul 09 04:46:48.365472 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.365433 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5f3e447-0444-419a-a3e8-840655333cd2-istio-podinfo\") pod \"a5f3e447-0444-419a-a3e8-840655333cd2\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " Jul 09 04:46:48.365472 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.365461 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-istio-data\") pod \"a5f3e447-0444-419a-a3e8-840655333cd2\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " Jul 09 04:46:48.365685 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.365496 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-credential-socket\") pod \"a5f3e447-0444-419a-a3e8-840655333cd2\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " Jul 09 04:46:48.365685 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.365506 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a5f3e447-0444-419a-a3e8-840655333cd2" (UID: "a5f3e447-0444-419a-a3e8-840655333cd2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:46:48.365685 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.365530 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-workload-socket\") pod \"a5f3e447-0444-419a-a3e8-840655333cd2\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " Jul 09 04:46:48.365685 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.365588 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-istio-envoy\") pod \"a5f3e447-0444-419a-a3e8-840655333cd2\" (UID: \"a5f3e447-0444-419a-a3e8-840655333cd2\") " Jul 09 04:46:48.365899 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.365761 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a5f3e447-0444-419a-a3e8-840655333cd2" (UID: "a5f3e447-0444-419a-a3e8-840655333cd2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:46:48.366063 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.365979 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a5f3e447-0444-419a-a3e8-840655333cd2" (UID: "a5f3e447-0444-419a-a3e8-840655333cd2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:46:48.366386 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.366366 2534 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-workload-certs\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:46:48.366549 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.366526 2534 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-credential-socket\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:46:48.366549 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.366555 2534 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-workload-socket\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:46:48.366549 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.366394 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5f3e447-0444-419a-a3e8-840655333cd2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a5f3e447-0444-419a-a3e8-840655333cd2" (UID: "a5f3e447-0444-419a-a3e8-840655333cd2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:46:48.366549 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.366391 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-istio-data" (OuterVolumeSpecName: "istio-data") pod "a5f3e447-0444-419a-a3e8-840655333cd2" (UID: "a5f3e447-0444-419a-a3e8-840655333cd2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:46:48.368099 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.368071 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a5f3e447-0444-419a-a3e8-840655333cd2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a5f3e447-0444-419a-a3e8-840655333cd2" (UID: "a5f3e447-0444-419a-a3e8-840655333cd2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 04:46:48.368099 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.368086 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5f3e447-0444-419a-a3e8-840655333cd2-istio-token" (OuterVolumeSpecName: "istio-token") pod "a5f3e447-0444-419a-a3e8-840655333cd2" (UID: "a5f3e447-0444-419a-a3e8-840655333cd2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:46:48.368250 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.368166 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5f3e447-0444-419a-a3e8-840655333cd2-kube-api-access-dj5v2" (OuterVolumeSpecName: "kube-api-access-dj5v2") pod "a5f3e447-0444-419a-a3e8-840655333cd2" (UID: "a5f3e447-0444-419a-a3e8-840655333cd2"). InnerVolumeSpecName "kube-api-access-dj5v2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:46:48.368250 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.368202 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a5f3e447-0444-419a-a3e8-840655333cd2" (UID: "a5f3e447-0444-419a-a3e8-840655333cd2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:46:48.467159 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.467127 2534 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-istio-envoy\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:46:48.467159 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.467153 2534 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5f3e447-0444-419a-a3e8-840655333cd2-istio-token\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:46:48.467159 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.467165 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dj5v2\" (UniqueName: \"kubernetes.io/projected/a5f3e447-0444-419a-a3e8-840655333cd2-kube-api-access-dj5v2\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:46:48.467418 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.467177 2534 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5f3e447-0444-419a-a3e8-840655333cd2-istiod-ca-cert\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:46:48.467418 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.467186 2534 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5f3e447-0444-419a-a3e8-840655333cd2-istio-podinfo\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:46:48.467418 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.467193 2534 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5f3e447-0444-419a-a3e8-840655333cd2-istio-data\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:46:48.806275 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.806240 2534 generic.go:356] "Generic (PLEG): container finished" podID="a5f3e447-0444-419a-a3e8-840655333cd2" containerID="b49c653f592464588fa3b732a3bff82964e6dc61a2b59c44be0ecbefc504165c" exitCode=0 Jul 09 04:46:48.806474 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.806309 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" Jul 09 04:46:48.806474 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.806329 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" event={"ID":"a5f3e447-0444-419a-a3e8-840655333cd2","Type":"ContainerDied","Data":"b49c653f592464588fa3b732a3bff82964e6dc61a2b59c44be0ecbefc504165c"} Jul 09 04:46:48.806474 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.806385 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz" event={"ID":"a5f3e447-0444-419a-a3e8-840655333cd2","Type":"ContainerDied","Data":"b6ed550e0d82c2c89776d7e983cb6a687401e3cad145df205e6cb7afc4e25a7c"} Jul 09 04:46:48.806474 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.806408 2534 scope.go:117] "RemoveContainer" containerID="b49c653f592464588fa3b732a3bff82964e6dc61a2b59c44be0ecbefc504165c" Jul 09 04:46:48.816171 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.816144 2534 scope.go:117] "RemoveContainer" containerID="b49c653f592464588fa3b732a3bff82964e6dc61a2b59c44be0ecbefc504165c" Jul 09 04:46:48.816452 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:46:48.816430 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b49c653f592464588fa3b732a3bff82964e6dc61a2b59c44be0ecbefc504165c\": container with ID starting with b49c653f592464588fa3b732a3bff82964e6dc61a2b59c44be0ecbefc504165c not found: ID does not exist" containerID="b49c653f592464588fa3b732a3bff82964e6dc61a2b59c44be0ecbefc504165c" Jul 09 04:46:48.816555 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.816459 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b49c653f592464588fa3b732a3bff82964e6dc61a2b59c44be0ecbefc504165c"} err="failed to get container status \"b49c653f592464588fa3b732a3bff82964e6dc61a2b59c44be0ecbefc504165c\": rpc error: code = NotFound desc = could not find container \"b49c653f592464588fa3b732a3bff82964e6dc61a2b59c44be0ecbefc504165c\": container with ID starting with b49c653f592464588fa3b732a3bff82964e6dc61a2b59c44be0ecbefc504165c not found: ID does not exist" Jul 09 04:46:48.826833 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.826807 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz"] Jul 09 04:46:48.830020 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:48.830001 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-d1630ae3-openshift-default-56689b5459-w79qz"] Jul 09 04:46:50.251682 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:50.251651 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5f3e447-0444-419a-a3e8-840655333cd2" path="/var/lib/kubelet/pods/a5f3e447-0444-419a-a3e8-840655333cd2/volumes" Jul 09 04:46:50.405146 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:50.405112 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8dnq_633a3ae5-3d78-4217-901a-6d0d06442cf4/console-operator/1.log" Jul 09 04:46:50.413140 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:46:50.413109 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8dnq_633a3ae5-3d78-4217-901a-6d0d06442cf4/console-operator/1.log" Jul 09 04:47:02.362371 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.362339 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm"] Jul 09 04:47:02.362798 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.362784 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aec72baa-af39-4268-921f-080fad640ca6" containerName="cleanup" Jul 09 04:47:02.362844 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.362800 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="aec72baa-af39-4268-921f-080fad640ca6" containerName="cleanup" Jul 09 04:47:02.362844 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.362819 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5f3e447-0444-419a-a3e8-840655333cd2" containerName="istio-proxy" Jul 09 04:47:02.362844 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.362824 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5f3e447-0444-419a-a3e8-840655333cd2" containerName="istio-proxy" Jul 09 04:47:02.362950 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.362919 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="aec72baa-af39-4268-921f-080fad640ca6" containerName="cleanup" Jul 09 04:47:02.362950 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.362933 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5f3e447-0444-419a-a3e8-840655333cd2" containerName="istio-proxy" Jul 09 04:47:02.366091 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.366071 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.369468 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.369446 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-f266a62c-openshift-default-dockercfg-kwh8r\"" Jul 09 04:47:02.396949 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.396918 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm"] Jul 09 04:47:02.498921 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.498882 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74165127-175a-4eaa-9ed7-cb9995d2cc82-istiod-ca-cert\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.499108 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.498941 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-workload-socket\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.499108 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.499063 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q92v4\" (UniqueName: \"kubernetes.io/projected/74165127-175a-4eaa-9ed7-cb9995d2cc82-kube-api-access-q92v4\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.499196 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.499129 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-workload-certs\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.499196 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.499161 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-istio-data\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.499196 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.499181 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74165127-175a-4eaa-9ed7-cb9995d2cc82-istio-token\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.499385 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.499206 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-istio-envoy\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.499385 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.499262 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-credential-socket\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.499385 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.499312 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74165127-175a-4eaa-9ed7-cb9995d2cc82-istio-podinfo\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.600209 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.600164 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-workload-socket\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.600209 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.600219 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q92v4\" (UniqueName: \"kubernetes.io/projected/74165127-175a-4eaa-9ed7-cb9995d2cc82-kube-api-access-q92v4\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.600533 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.600266 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-workload-certs\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.600533 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.600317 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-istio-data\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.600533 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.600347 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74165127-175a-4eaa-9ed7-cb9995d2cc82-istio-token\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.600533 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.600372 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-istio-envoy\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.600533 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.600420 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-credential-socket\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.600533 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.600457 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74165127-175a-4eaa-9ed7-cb9995d2cc82-istio-podinfo\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.600533 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.600481 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74165127-175a-4eaa-9ed7-cb9995d2cc82-istiod-ca-cert\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.600836 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.600700 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-istio-data\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.600894 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.600871 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-workload-socket\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.600972 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.600945 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-credential-socket\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.601208 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.601166 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-workload-certs\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.601389 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.601323 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74165127-175a-4eaa-9ed7-cb9995d2cc82-istiod-ca-cert\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.603604 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.603579 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-istio-envoy\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.603762 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.603746 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74165127-175a-4eaa-9ed7-cb9995d2cc82-istio-podinfo\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.608095 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.608052 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q92v4\" (UniqueName: \"kubernetes.io/projected/74165127-175a-4eaa-9ed7-cb9995d2cc82-kube-api-access-q92v4\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.608178 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.608070 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74165127-175a-4eaa-9ed7-cb9995d2cc82-istio-token\") pod \"e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.676210 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.676136 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:02.814609 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.814583 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm"] Jul 09 04:47:02.816899 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:47:02.816871 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74165127_175a_4eaa_9ed7_cb9995d2cc82.slice/crio-992cccebb0b1621738869b6b6f968f4b40d5066f62109096c8e62f370ba8a9ba WatchSource:0}: Error finding container 992cccebb0b1621738869b6b6f968f4b40d5066f62109096c8e62f370ba8a9ba: Status 404 returned error can't find the container with id 992cccebb0b1621738869b6b6f968f4b40d5066f62109096c8e62f370ba8a9ba Jul 09 04:47:02.819437 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.819408 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:47:02.819522 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.819472 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:47:02.819522 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.819501 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:47:02.880944 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:02.880919 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" event={"ID":"74165127-175a-4eaa-9ed7-cb9995d2cc82","Type":"ContainerStarted","Data":"992cccebb0b1621738869b6b6f968f4b40d5066f62109096c8e62f370ba8a9ba"} Jul 09 04:47:03.887185 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:03.887144 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" event={"ID":"74165127-175a-4eaa-9ed7-cb9995d2cc82","Type":"ContainerStarted","Data":"0037211d3132a9051c16fea730f2a5be54bd4909ac13bf550ecb17d615a94cd0"} Jul 09 04:47:03.906417 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:03.906363 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" podStartSLOduration=1.9063435389999999 podStartE2EDuration="1.906343539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:47:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:47:03.905483837 +0000 UTC m=+2414.129102325" watchObservedRunningTime="2026-07-09 04:47:03.906343539 +0000 UTC m=+2414.129962028" Jul 09 04:47:04.676953 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:04.676918 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:04.681871 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:04.681838 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:04.892847 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:04.892805 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:47:04.893757 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:47:04.893738 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:50:09.781780 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:09.781749 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm"] Jul 09 04:50:09.782338 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:09.781971 2534 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" podUID="74165127-175a-4eaa-9ed7-cb9995d2cc82" containerName="istio-proxy" containerID="cri-o://0037211d3132a9051c16fea730f2a5be54bd4909ac13bf550ecb17d615a94cd0" gracePeriod=30 Jul 09 04:50:15.040437 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.040370 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:50:15.127802 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.127775 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74165127-175a-4eaa-9ed7-cb9995d2cc82-istio-token\") pod \"74165127-175a-4eaa-9ed7-cb9995d2cc82\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " Jul 09 04:50:15.127971 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.127816 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-istio-envoy\") pod \"74165127-175a-4eaa-9ed7-cb9995d2cc82\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " Jul 09 04:50:15.127971 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.127845 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-credential-socket\") pod \"74165127-175a-4eaa-9ed7-cb9995d2cc82\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " Jul 09 04:50:15.127971 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.127877 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-istio-data\") pod \"74165127-175a-4eaa-9ed7-cb9995d2cc82\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " Jul 09 04:50:15.127971 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.127903 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74165127-175a-4eaa-9ed7-cb9995d2cc82-istio-podinfo\") pod \"74165127-175a-4eaa-9ed7-cb9995d2cc82\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " Jul 09 04:50:15.127971 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.127924 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74165127-175a-4eaa-9ed7-cb9995d2cc82-istiod-ca-cert\") pod \"74165127-175a-4eaa-9ed7-cb9995d2cc82\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " Jul 09 04:50:15.127971 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.127945 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-workload-socket\") pod \"74165127-175a-4eaa-9ed7-cb9995d2cc82\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " Jul 09 04:50:15.127971 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.127970 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q92v4\" (UniqueName: \"kubernetes.io/projected/74165127-175a-4eaa-9ed7-cb9995d2cc82-kube-api-access-q92v4\") pod \"74165127-175a-4eaa-9ed7-cb9995d2cc82\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " Jul 09 04:50:15.128356 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.128259 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-istio-data" (OuterVolumeSpecName: "istio-data") pod "74165127-175a-4eaa-9ed7-cb9995d2cc82" (UID: "74165127-175a-4eaa-9ed7-cb9995d2cc82"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:50:15.128418 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.128349 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "74165127-175a-4eaa-9ed7-cb9995d2cc82" (UID: "74165127-175a-4eaa-9ed7-cb9995d2cc82"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:50:15.128418 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.128383 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-workload-certs\") pod \"74165127-175a-4eaa-9ed7-cb9995d2cc82\" (UID: \"74165127-175a-4eaa-9ed7-cb9995d2cc82\") " Jul 09 04:50:15.128526 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.128390 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "74165127-175a-4eaa-9ed7-cb9995d2cc82" (UID: "74165127-175a-4eaa-9ed7-cb9995d2cc82"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:50:15.128589 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.128504 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74165127-175a-4eaa-9ed7-cb9995d2cc82-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "74165127-175a-4eaa-9ed7-cb9995d2cc82" (UID: "74165127-175a-4eaa-9ed7-cb9995d2cc82"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:50:15.128589 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.128571 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "74165127-175a-4eaa-9ed7-cb9995d2cc82" (UID: "74165127-175a-4eaa-9ed7-cb9995d2cc82"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:50:15.128805 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.128758 2534 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-credential-socket\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:50:15.128805 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.128787 2534 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-istio-data\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:50:15.128805 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.128804 2534 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74165127-175a-4eaa-9ed7-cb9995d2cc82-istiod-ca-cert\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:50:15.129054 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.128818 2534 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-workload-socket\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:50:15.129054 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.128832 2534 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-workload-certs\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:50:15.130365 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.130344 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "74165127-175a-4eaa-9ed7-cb9995d2cc82" (UID: "74165127-175a-4eaa-9ed7-cb9995d2cc82"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:50:15.130592 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.130571 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74165127-175a-4eaa-9ed7-cb9995d2cc82-kube-api-access-q92v4" (OuterVolumeSpecName: "kube-api-access-q92v4") pod "74165127-175a-4eaa-9ed7-cb9995d2cc82" (UID: "74165127-175a-4eaa-9ed7-cb9995d2cc82"). InnerVolumeSpecName "kube-api-access-q92v4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:50:15.130782 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.130758 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/74165127-175a-4eaa-9ed7-cb9995d2cc82-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "74165127-175a-4eaa-9ed7-cb9995d2cc82" (UID: "74165127-175a-4eaa-9ed7-cb9995d2cc82"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 04:50:15.130847 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.130763 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74165127-175a-4eaa-9ed7-cb9995d2cc82-istio-token" (OuterVolumeSpecName: "istio-token") pod "74165127-175a-4eaa-9ed7-cb9995d2cc82" (UID: "74165127-175a-4eaa-9ed7-cb9995d2cc82"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:50:15.229434 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.229401 2534 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74165127-175a-4eaa-9ed7-cb9995d2cc82-istio-token\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:50:15.229434 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.229426 2534 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74165127-175a-4eaa-9ed7-cb9995d2cc82-istio-envoy\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:50:15.229434 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.229436 2534 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74165127-175a-4eaa-9ed7-cb9995d2cc82-istio-podinfo\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:50:15.229645 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.229444 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q92v4\" (UniqueName: \"kubernetes.io/projected/74165127-175a-4eaa-9ed7-cb9995d2cc82-kube-api-access-q92v4\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:50:15.871728 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.871693 2534 generic.go:356] "Generic (PLEG): container finished" podID="74165127-175a-4eaa-9ed7-cb9995d2cc82" containerID="0037211d3132a9051c16fea730f2a5be54bd4909ac13bf550ecb17d615a94cd0" exitCode=0 Jul 09 04:50:15.871908 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.871740 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" Jul 09 04:50:15.871908 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.871777 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" event={"ID":"74165127-175a-4eaa-9ed7-cb9995d2cc82","Type":"ContainerDied","Data":"0037211d3132a9051c16fea730f2a5be54bd4909ac13bf550ecb17d615a94cd0"} Jul 09 04:50:15.871908 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.871832 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm" event={"ID":"74165127-175a-4eaa-9ed7-cb9995d2cc82","Type":"ContainerDied","Data":"992cccebb0b1621738869b6b6f968f4b40d5066f62109096c8e62f370ba8a9ba"} Jul 09 04:50:15.871908 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.871854 2534 scope.go:117] "RemoveContainer" containerID="0037211d3132a9051c16fea730f2a5be54bd4909ac13bf550ecb17d615a94cd0" Jul 09 04:50:15.881377 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.881360 2534 scope.go:117] "RemoveContainer" containerID="0037211d3132a9051c16fea730f2a5be54bd4909ac13bf550ecb17d615a94cd0" Jul 09 04:50:15.881617 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:50:15.881601 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0037211d3132a9051c16fea730f2a5be54bd4909ac13bf550ecb17d615a94cd0\": container with ID starting with 0037211d3132a9051c16fea730f2a5be54bd4909ac13bf550ecb17d615a94cd0 not found: ID does not exist" containerID="0037211d3132a9051c16fea730f2a5be54bd4909ac13bf550ecb17d615a94cd0" Jul 09 04:50:15.881668 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.881624 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0037211d3132a9051c16fea730f2a5be54bd4909ac13bf550ecb17d615a94cd0"} err="failed to get container status \"0037211d3132a9051c16fea730f2a5be54bd4909ac13bf550ecb17d615a94cd0\": rpc error: code = NotFound desc = could not find container \"0037211d3132a9051c16fea730f2a5be54bd4909ac13bf550ecb17d615a94cd0\": container with ID starting with 0037211d3132a9051c16fea730f2a5be54bd4909ac13bf550ecb17d615a94cd0 not found: ID does not exist" Jul 09 04:50:15.893095 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.893075 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm"] Jul 09 04:50:15.896800 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:15.896778 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-f266a62c-openshift-default-9b659bbff-l7tpm"] Jul 09 04:50:16.250211 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:50:16.250133 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74165127-175a-4eaa-9ed7-cb9995d2cc82" path="/var/lib/kubelet/pods/74165127-175a-4eaa-9ed7-cb9995d2cc82/volumes" Jul 09 04:51:03.635520 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.635482 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp"] Jul 09 04:51:03.636172 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.636147 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74165127-175a-4eaa-9ed7-cb9995d2cc82" containerName="istio-proxy" Jul 09 04:51:03.636172 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.636176 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="74165127-175a-4eaa-9ed7-cb9995d2cc82" containerName="istio-proxy" Jul 09 04:51:03.636343 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.636271 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="74165127-175a-4eaa-9ed7-cb9995d2cc82" containerName="istio-proxy" Jul 09 04:51:03.639814 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.639798 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.642080 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.642061 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-4a3e5d-openshift-default-dockercfg-nrld9\"" Jul 09 04:51:03.652848 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.652824 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp"] Jul 09 04:51:03.690951 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.690917 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-credential-socket\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.690951 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.690955 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/033432a7-7c0e-464e-b43b-b557918db184-istio-podinfo\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.691180 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.691005 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-workload-socket\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.691180 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.691044 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/033432a7-7c0e-464e-b43b-b557918db184-istio-token\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.691319 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.691180 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-workload-certs\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.691319 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.691211 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-istio-envoy\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.691319 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.691277 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-istio-data\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.691479 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.691337 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/033432a7-7c0e-464e-b43b-b557918db184-istiod-ca-cert\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.691479 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.691383 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5c48\" (UniqueName: \"kubernetes.io/projected/033432a7-7c0e-464e-b43b-b557918db184-kube-api-access-g5c48\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.792198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.792160 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/033432a7-7c0e-464e-b43b-b557918db184-istio-podinfo\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.792198 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.792200 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-workload-socket\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.792492 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.792221 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/033432a7-7c0e-464e-b43b-b557918db184-istio-token\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.792492 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.792325 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-workload-certs\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.792492 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.792348 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-istio-envoy\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.792492 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.792392 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-istio-data\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.792492 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.792413 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/033432a7-7c0e-464e-b43b-b557918db184-istiod-ca-cert\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.792492 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.792452 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5c48\" (UniqueName: \"kubernetes.io/projected/033432a7-7c0e-464e-b43b-b557918db184-kube-api-access-g5c48\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.792807 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.792503 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-credential-socket\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.792807 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.792701 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-workload-socket\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.792807 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.792751 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-workload-certs\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.792807 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.792783 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-istio-data\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.793024 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.792895 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-credential-socket\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.793222 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.793193 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/033432a7-7c0e-464e-b43b-b557918db184-istiod-ca-cert\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.794869 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.794842 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-istio-envoy\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.795331 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.795313 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/033432a7-7c0e-464e-b43b-b557918db184-istio-podinfo\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.799829 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.799807 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/033432a7-7c0e-464e-b43b-b557918db184-istio-token\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.800021 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.800000 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5c48\" (UniqueName: \"kubernetes.io/projected/033432a7-7c0e-464e-b43b-b557918db184-kube-api-access-g5c48\") pod \"e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:03.951431 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:03.951355 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:04.098071 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:04.098036 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp"] Jul 09 04:51:04.101534 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:51:04.101500 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod033432a7_7c0e_464e_b43b_b557918db184.slice/crio-efa496c1a62925d83b82d7c7e0729d9ada9713548504d846af400a6237af21b9 WatchSource:0}: Error finding container efa496c1a62925d83b82d7c7e0729d9ada9713548504d846af400a6237af21b9: Status 404 returned error can't find the container with id efa496c1a62925d83b82d7c7e0729d9ada9713548504d846af400a6237af21b9 Jul 09 04:51:04.103617 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:04.103583 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:51:04.103704 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:04.103649 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:51:04.103704 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:04.103695 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:51:04.114174 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:04.114147 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" event={"ID":"033432a7-7c0e-464e-b43b-b557918db184","Type":"ContainerStarted","Data":"efa496c1a62925d83b82d7c7e0729d9ada9713548504d846af400a6237af21b9"} Jul 09 04:51:05.120538 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:05.120500 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" event={"ID":"033432a7-7c0e-464e-b43b-b557918db184","Type":"ContainerStarted","Data":"678b49ed636383787b079ba8662fad637487e9c9317119c193a128a6c4100cc2"} Jul 09 04:51:05.138886 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:05.138843 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" podStartSLOduration=2.138827285 podStartE2EDuration="2.138827285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:51:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:51:05.137465066 +0000 UTC m=+2655.361083553" watchObservedRunningTime="2026-07-09 04:51:05.138827285 +0000 UTC m=+2655.362445772" Jul 09 04:51:05.952565 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:05.952529 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:05.957273 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:05.957243 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:06.125958 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:06.125929 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:06.126968 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:06.126948 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:51:50.437139 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:50.437056 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8dnq_633a3ae5-3d78-4217-901a-6d0d06442cf4/console-operator/1.log" Jul 09 04:51:50.445429 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:51:50.445404 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8dnq_633a3ae5-3d78-4217-901a-6d0d06442cf4/console-operator/1.log" Jul 09 04:54:11.056334 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:11.056277 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp"] Jul 09 04:54:11.056901 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:11.056571 2534 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" podUID="033432a7-7c0e-464e-b43b-b557918db184" containerName="istio-proxy" containerID="cri-o://678b49ed636383787b079ba8662fad637487e9c9317119c193a128a6c4100cc2" gracePeriod=30 Jul 09 04:54:16.308424 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.308401 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:54:16.408646 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.408570 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/033432a7-7c0e-464e-b43b-b557918db184-istio-podinfo\") pod \"033432a7-7c0e-464e-b43b-b557918db184\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " Jul 09 04:54:16.408646 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.408602 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/033432a7-7c0e-464e-b43b-b557918db184-istiod-ca-cert\") pod \"033432a7-7c0e-464e-b43b-b557918db184\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " Jul 09 04:54:16.408646 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.408619 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-workload-socket\") pod \"033432a7-7c0e-464e-b43b-b557918db184\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " Jul 09 04:54:16.408646 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.408637 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/033432a7-7c0e-464e-b43b-b557918db184-istio-token\") pod \"033432a7-7c0e-464e-b43b-b557918db184\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " Jul 09 04:54:16.408953 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.408664 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-istio-data\") pod \"033432a7-7c0e-464e-b43b-b557918db184\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " Jul 09 04:54:16.408953 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.408705 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5c48\" (UniqueName: \"kubernetes.io/projected/033432a7-7c0e-464e-b43b-b557918db184-kube-api-access-g5c48\") pod \"033432a7-7c0e-464e-b43b-b557918db184\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " Jul 09 04:54:16.408953 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.408741 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-workload-certs\") pod \"033432a7-7c0e-464e-b43b-b557918db184\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " Jul 09 04:54:16.408953 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.408785 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-istio-envoy\") pod \"033432a7-7c0e-464e-b43b-b557918db184\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " Jul 09 04:54:16.408953 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.408829 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-credential-socket\") pod \"033432a7-7c0e-464e-b43b-b557918db184\" (UID: \"033432a7-7c0e-464e-b43b-b557918db184\") " Jul 09 04:54:16.408953 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.408935 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "033432a7-7c0e-464e-b43b-b557918db184" (UID: "033432a7-7c0e-464e-b43b-b557918db184"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:54:16.409253 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.408987 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-istio-data" (OuterVolumeSpecName: "istio-data") pod "033432a7-7c0e-464e-b43b-b557918db184" (UID: "033432a7-7c0e-464e-b43b-b557918db184"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:54:16.409253 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.409042 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "033432a7-7c0e-464e-b43b-b557918db184" (UID: "033432a7-7c0e-464e-b43b-b557918db184"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:54:16.409253 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.409112 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/033432a7-7c0e-464e-b43b-b557918db184-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "033432a7-7c0e-464e-b43b-b557918db184" (UID: "033432a7-7c0e-464e-b43b-b557918db184"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:54:16.409253 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.409224 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "033432a7-7c0e-464e-b43b-b557918db184" (UID: "033432a7-7c0e-464e-b43b-b557918db184"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:54:16.409505 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.409302 2534 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-workload-certs\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:54:16.409505 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.409323 2534 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/033432a7-7c0e-464e-b43b-b557918db184-istiod-ca-cert\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:54:16.409505 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.409332 2534 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-workload-socket\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:54:16.409505 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.409345 2534 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-istio-data\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:54:16.411001 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.410963 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/033432a7-7c0e-464e-b43b-b557918db184-istio-token" (OuterVolumeSpecName: "istio-token") pod "033432a7-7c0e-464e-b43b-b557918db184" (UID: "033432a7-7c0e-464e-b43b-b557918db184"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:54:16.411113 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.411035 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "033432a7-7c0e-464e-b43b-b557918db184" (UID: "033432a7-7c0e-464e-b43b-b557918db184"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:54:16.411113 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.411035 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/033432a7-7c0e-464e-b43b-b557918db184-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "033432a7-7c0e-464e-b43b-b557918db184" (UID: "033432a7-7c0e-464e-b43b-b557918db184"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 04:54:16.411359 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.411338 2534 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/033432a7-7c0e-464e-b43b-b557918db184-kube-api-access-g5c48" (OuterVolumeSpecName: "kube-api-access-g5c48") pod "033432a7-7c0e-464e-b43b-b557918db184" (UID: "033432a7-7c0e-464e-b43b-b557918db184"). InnerVolumeSpecName "kube-api-access-g5c48". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:54:16.509967 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.509936 2534 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-istio-envoy\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:54:16.509967 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.509964 2534 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/033432a7-7c0e-464e-b43b-b557918db184-credential-socket\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:54:16.510134 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.509979 2534 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/033432a7-7c0e-464e-b43b-b557918db184-istio-podinfo\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:54:16.510134 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.509992 2534 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/033432a7-7c0e-464e-b43b-b557918db184-istio-token\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:54:16.510134 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:16.510004 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5c48\" (UniqueName: \"kubernetes.io/projected/033432a7-7c0e-464e-b43b-b557918db184-kube-api-access-g5c48\") on node \"ip-10-0-130-153.ec2.internal\" DevicePath \"\"" Jul 09 04:54:17.095628 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:17.095595 2534 generic.go:356] "Generic (PLEG): container finished" podID="033432a7-7c0e-464e-b43b-b557918db184" containerID="678b49ed636383787b079ba8662fad637487e9c9317119c193a128a6c4100cc2" exitCode=0 Jul 09 04:54:17.095808 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:17.095647 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" Jul 09 04:54:17.095808 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:17.095680 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" event={"ID":"033432a7-7c0e-464e-b43b-b557918db184","Type":"ContainerDied","Data":"678b49ed636383787b079ba8662fad637487e9c9317119c193a128a6c4100cc2"} Jul 09 04:54:17.095808 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:17.095737 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp" event={"ID":"033432a7-7c0e-464e-b43b-b557918db184","Type":"ContainerDied","Data":"efa496c1a62925d83b82d7c7e0729d9ada9713548504d846af400a6237af21b9"} Jul 09 04:54:17.095808 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:17.095766 2534 scope.go:117] "RemoveContainer" containerID="678b49ed636383787b079ba8662fad637487e9c9317119c193a128a6c4100cc2" Jul 09 04:54:17.105683 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:17.105661 2534 scope.go:117] "RemoveContainer" containerID="678b49ed636383787b079ba8662fad637487e9c9317119c193a128a6c4100cc2" Jul 09 04:54:17.105936 ip-10-0-130-153 kubenswrapper[2534]: E0709 04:54:17.105915 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"678b49ed636383787b079ba8662fad637487e9c9317119c193a128a6c4100cc2\": container with ID starting with 678b49ed636383787b079ba8662fad637487e9c9317119c193a128a6c4100cc2 not found: ID does not exist" containerID="678b49ed636383787b079ba8662fad637487e9c9317119c193a128a6c4100cc2" Jul 09 04:54:17.105990 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:17.105943 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"678b49ed636383787b079ba8662fad637487e9c9317119c193a128a6c4100cc2"} err="failed to get container status \"678b49ed636383787b079ba8662fad637487e9c9317119c193a128a6c4100cc2\": rpc error: code = NotFound desc = could not find container \"678b49ed636383787b079ba8662fad637487e9c9317119c193a128a6c4100cc2\": container with ID starting with 678b49ed636383787b079ba8662fad637487e9c9317119c193a128a6c4100cc2 not found: ID does not exist" Jul 09 04:54:17.117523 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:17.117502 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp"] Jul 09 04:54:17.120236 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:17.120215 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-4a3e5d-openshift-default-c497c4c85-d5ptp"] Jul 09 04:54:17.521561 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:17.521480 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-f7cf9bf97-pmb2x_01c7c3c4-6115-404b-815b-058927f121a1/manager/0.log" Jul 09 04:54:17.969703 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:17.969679 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cff547d94-vs2gg_5a2d3035-3a6a-4db4-bc3f-de7a6c360ded/copy-manifests/0.log" Jul 09 04:54:17.987112 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:17.987081 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cff547d94-vs2gg_5a2d3035-3a6a-4db4-bc3f-de7a6c360ded/manager/0.log" Jul 09 04:54:18.251008 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:18.250920 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="033432a7-7c0e-464e-b43b-b557918db184" path="/var/lib/kubelet/pods/033432a7-7c0e-464e-b43b-b557918db184/volumes" Jul 09 04:54:18.441040 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:18.441008 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5b664864d4-ls5q8_59a544dc-35c1-4fb9-9b8f-d30d9c268d60/maas-api/0.log" Jul 09 04:54:18.548883 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:18.548854 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29726190-cb6pc_1eb4ac7a-4a12-4304-98b7-c1f1b95061a3/cleanup/0.log" Jul 09 04:54:18.661692 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:18.661669 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29726205-fqkxl_aec72baa-af39-4268-921f-080fad640ca6/cleanup/0.log" Jul 09 04:54:18.768852 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:18.768822 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-4rdq8_983f5f4c-a908-4b8d-a24f-87f231e6b5dd/postgres/0.log" Jul 09 04:54:21.314092 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:21.314012 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6wqjx_90f3ac8b-482f-439f-ae20-05262ae79c22/discovery/0.log" Jul 09 04:54:21.652441 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:21.652360 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-khl6s_136f41c8-89e3-4c37-9bf8-e4d68b96a781/istio-proxy/1.log" Jul 09 04:54:21.766348 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:21.766308 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-v9b9w_54b0094b-edbc-4be5-8c85-b6dbed9171cf/payload-pre-processing/0.log" Jul 09 04:54:21.886586 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:21.886559 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-prdjf_51c51e26-4ee8-4759-9885-2d7c389c662d/payload-processing/0.log" Jul 09 04:54:22.329482 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:22.329453 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-sbbc2_9236344a-f135-4188-919c-59061b63d8e3/main/0.log" Jul 09 04:54:22.438707 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:22.438676 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-mz8hc_c5c62ba3-ff83-42ee-9deb-e3396f084dda/main/0.log" Jul 09 04:54:29.946303 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:29.946247 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-krxxf_2c0aaa25-8a07-4da9-8d7f-b8abd6026f36/global-pull-secret-syncer/0.log" Jul 09 04:54:30.034029 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:30.033992 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xksp2_e6cff921-2b0d-4748-ab0f-642fc9ff3d7e/konnectivity-agent/0.log" Jul 09 04:54:30.104626 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:30.104598 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-153.ec2.internal_67b85ef8254593a9d9a03d6a7f9822a9/haproxy/0.log" Jul 09 04:54:37.958440 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:37.958411 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-4cnc4_742d0176-a6e5-4a5a-932b-460367548241/kube-state-metrics/0.log" Jul 09 04:54:37.979089 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:37.979065 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-4cnc4_742d0176-a6e5-4a5a-932b-460367548241/kube-rbac-proxy-main/0.log" Jul 09 04:54:37.998380 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:37.998359 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-4cnc4_742d0176-a6e5-4a5a-932b-460367548241/kube-rbac-proxy-self/0.log" Jul 09 04:54:38.251064 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:38.251027 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vljg2_0a316c28-7d81-4e36-bff6-7ea5ef5ab248/node-exporter/0.log" Jul 09 04:54:38.271866 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:38.271842 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vljg2_0a316c28-7d81-4e36-bff6-7ea5ef5ab248/kube-rbac-proxy/0.log" Jul 09 04:54:38.292134 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:38.292113 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vljg2_0a316c28-7d81-4e36-bff6-7ea5ef5ab248/init-textfile/0.log" Jul 09 04:54:38.318771 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:38.318740 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-xrj2l_52f790aa-4d62-4d1f-afee-2fb4aa845265/kube-rbac-proxy-main/0.log" Jul 09 04:54:38.340068 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:38.340036 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-xrj2l_52f790aa-4d62-4d1f-afee-2fb4aa845265/kube-rbac-proxy-self/0.log" Jul 09 04:54:38.363073 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:38.363054 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-xrj2l_52f790aa-4d62-4d1f-afee-2fb4aa845265/openshift-state-metrics/0.log" Jul 09 04:54:38.398397 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:38.398371 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c1252f86-581d-4394-a3ac-d34d38b772e0/prometheus/0.log" Jul 09 04:54:38.417620 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:38.417580 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c1252f86-581d-4394-a3ac-d34d38b772e0/config-reloader/0.log" Jul 09 04:54:38.437418 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:38.437395 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c1252f86-581d-4394-a3ac-d34d38b772e0/thanos-sidecar/0.log" Jul 09 04:54:38.472551 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:38.472530 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c1252f86-581d-4394-a3ac-d34d38b772e0/kube-rbac-proxy-web/0.log" Jul 09 04:54:38.492563 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:38.492542 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c1252f86-581d-4394-a3ac-d34d38b772e0/kube-rbac-proxy/0.log" Jul 09 04:54:38.512536 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:38.512472 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c1252f86-581d-4394-a3ac-d34d38b772e0/kube-rbac-proxy-thanos/0.log" Jul 09 04:54:38.532536 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:38.532514 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c1252f86-581d-4394-a3ac-d34d38b772e0/init-config-reloader/0.log" Jul 09 04:54:38.560523 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:38.560501 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-nzn6j_81de11f1-ca34-4c0a-be5d-5ff455159cbb/prometheus-operator/0.log" Jul 09 04:54:38.577463 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:38.577435 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-nzn6j_81de11f1-ca34-4c0a-be5d-5ff455159cbb/kube-rbac-proxy/0.log" Jul 09 04:54:38.712578 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:38.712542 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8fd4cf69b-c4gtr_3fa09ac0-300c-4d25-baa5-e07c192168e8/thanos-query/0.log" Jul 09 04:54:38.731912 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:38.731886 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8fd4cf69b-c4gtr_3fa09ac0-300c-4d25-baa5-e07c192168e8/kube-rbac-proxy-web/0.log" Jul 09 04:54:38.751173 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:38.751133 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8fd4cf69b-c4gtr_3fa09ac0-300c-4d25-baa5-e07c192168e8/kube-rbac-proxy/0.log" Jul 09 04:54:38.770334 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:38.770261 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8fd4cf69b-c4gtr_3fa09ac0-300c-4d25-baa5-e07c192168e8/prom-label-proxy/0.log" Jul 09 04:54:38.799521 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:38.799503 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8fd4cf69b-c4gtr_3fa09ac0-300c-4d25-baa5-e07c192168e8/kube-rbac-proxy-rules/0.log" Jul 09 04:54:38.820299 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:38.820271 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8fd4cf69b-c4gtr_3fa09ac0-300c-4d25-baa5-e07c192168e8/kube-rbac-proxy-metrics/0.log" Jul 09 04:54:39.926682 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:39.926649 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vrs45/perf-node-gather-daemonset-kdq86"] Jul 09 04:54:39.927299 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:39.927263 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="033432a7-7c0e-464e-b43b-b557918db184" containerName="istio-proxy" Jul 09 04:54:39.927369 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:39.927302 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="033432a7-7c0e-464e-b43b-b557918db184" containerName="istio-proxy" Jul 09 04:54:39.927417 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:39.927403 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="033432a7-7c0e-464e-b43b-b557918db184" containerName="istio-proxy" Jul 09 04:54:39.930747 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:39.930728 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-kdq86" Jul 09 04:54:39.933259 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:39.933242 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vrs45\"/\"openshift-service-ca.crt\"" Jul 09 04:54:39.934438 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:39.934416 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vrs45\"/\"kube-root-ca.crt\"" Jul 09 04:54:39.934564 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:39.934426 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vrs45\"/\"default-dockercfg-x6rg7\"" Jul 09 04:54:39.937687 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:39.937663 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vrs45/perf-node-gather-daemonset-kdq86"] Jul 09 04:54:39.952135 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:39.952115 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-z5c2w_9d2284ee-ac3c-4e3c-9b08-67abc9f14e75/networking-console-plugin/0.log" Jul 09 04:54:40.024478 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:40.024457 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/117e290c-6ab5-4822-8212-5f637f523f3f-podres\") pod \"perf-node-gather-daemonset-kdq86\" (UID: \"117e290c-6ab5-4822-8212-5f637f523f3f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-kdq86" Jul 09 04:54:40.024584 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:40.024500 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/117e290c-6ab5-4822-8212-5f637f523f3f-proc\") pod \"perf-node-gather-daemonset-kdq86\" (UID: \"117e290c-6ab5-4822-8212-5f637f523f3f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-kdq86" Jul 09 04:54:40.024584 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:40.024527 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsmhw\" (UniqueName: \"kubernetes.io/projected/117e290c-6ab5-4822-8212-5f637f523f3f-kube-api-access-qsmhw\") pod \"perf-node-gather-daemonset-kdq86\" (UID: \"117e290c-6ab5-4822-8212-5f637f523f3f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-kdq86" Jul 09 04:54:40.024667 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:40.024626 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/117e290c-6ab5-4822-8212-5f637f523f3f-sys\") pod \"perf-node-gather-daemonset-kdq86\" (UID: \"117e290c-6ab5-4822-8212-5f637f523f3f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-kdq86" Jul 09 04:54:40.024667 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:40.024655 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/117e290c-6ab5-4822-8212-5f637f523f3f-lib-modules\") pod \"perf-node-gather-daemonset-kdq86\" (UID: \"117e290c-6ab5-4822-8212-5f637f523f3f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-kdq86" Jul 09 04:54:40.125670 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:40.125642 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/117e290c-6ab5-4822-8212-5f637f523f3f-podres\") pod \"perf-node-gather-daemonset-kdq86\" (UID: \"117e290c-6ab5-4822-8212-5f637f523f3f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-kdq86" Jul 09 04:54:40.125816 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:40.125703 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/117e290c-6ab5-4822-8212-5f637f523f3f-proc\") pod \"perf-node-gather-daemonset-kdq86\" (UID: \"117e290c-6ab5-4822-8212-5f637f523f3f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-kdq86" Jul 09 04:54:40.125816 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:40.125754 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsmhw\" (UniqueName: \"kubernetes.io/projected/117e290c-6ab5-4822-8212-5f637f523f3f-kube-api-access-qsmhw\") pod \"perf-node-gather-daemonset-kdq86\" (UID: \"117e290c-6ab5-4822-8212-5f637f523f3f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-kdq86" Jul 09 04:54:40.125816 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:40.125781 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/117e290c-6ab5-4822-8212-5f637f523f3f-proc\") pod \"perf-node-gather-daemonset-kdq86\" (UID: \"117e290c-6ab5-4822-8212-5f637f523f3f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-kdq86" Jul 09 04:54:40.126023 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:40.125819 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/117e290c-6ab5-4822-8212-5f637f523f3f-podres\") pod \"perf-node-gather-daemonset-kdq86\" (UID: \"117e290c-6ab5-4822-8212-5f637f523f3f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-kdq86" Jul 09 04:54:40.126023 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:40.125849 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/117e290c-6ab5-4822-8212-5f637f523f3f-sys\") pod \"perf-node-gather-daemonset-kdq86\" (UID: \"117e290c-6ab5-4822-8212-5f637f523f3f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-kdq86" Jul 09 04:54:40.126023 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:40.125878 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/117e290c-6ab5-4822-8212-5f637f523f3f-lib-modules\") pod \"perf-node-gather-daemonset-kdq86\" (UID: \"117e290c-6ab5-4822-8212-5f637f523f3f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-kdq86" Jul 09 04:54:40.126023 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:40.125968 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/117e290c-6ab5-4822-8212-5f637f523f3f-sys\") pod \"perf-node-gather-daemonset-kdq86\" (UID: \"117e290c-6ab5-4822-8212-5f637f523f3f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-kdq86" Jul 09 04:54:40.126153 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:40.126028 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/117e290c-6ab5-4822-8212-5f637f523f3f-lib-modules\") pod \"perf-node-gather-daemonset-kdq86\" (UID: \"117e290c-6ab5-4822-8212-5f637f523f3f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-kdq86" Jul 09 04:54:40.133125 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:40.133103 2534 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsmhw\" (UniqueName: \"kubernetes.io/projected/117e290c-6ab5-4822-8212-5f637f523f3f-kube-api-access-qsmhw\") pod \"perf-node-gather-daemonset-kdq86\" (UID: \"117e290c-6ab5-4822-8212-5f637f523f3f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-kdq86" Jul 09 04:54:40.243404 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:40.243339 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-kdq86" Jul 09 04:54:40.379571 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:40.379548 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vrs45/perf-node-gather-daemonset-kdq86"] Jul 09 04:54:40.381637 ip-10-0-130-153 kubenswrapper[2534]: W0709 04:54:40.381610 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod117e290c_6ab5_4822_8212_5f637f523f3f.slice/crio-b8ef2ba7a65be85b95728147162ae701b9635c4bf543e26fd9b0fe14a2585991 WatchSource:0}: Error finding container b8ef2ba7a65be85b95728147162ae701b9635c4bf543e26fd9b0fe14a2585991: Status 404 returned error can't find the container with id b8ef2ba7a65be85b95728147162ae701b9635c4bf543e26fd9b0fe14a2585991 Jul 09 04:54:40.586854 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:40.586824 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8dnq_633a3ae5-3d78-4217-901a-6d0d06442cf4/console-operator/1.log" Jul 09 04:54:40.595362 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:40.595341 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8dnq_633a3ae5-3d78-4217-901a-6d0d06442cf4/console-operator/2.log" Jul 09 04:54:41.085354 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:41.085325 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-6q5mp_e539f6ff-e687-4517-b313-96a208ec1d48/download-server/0.log" Jul 09 04:54:41.222839 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:41.222807 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-kdq86" event={"ID":"117e290c-6ab5-4822-8212-5f637f523f3f","Type":"ContainerStarted","Data":"90cfe44f5e957894434348a012c04cb178296118cb9c366639694411b169bb91"} Jul 09 04:54:41.223034 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:41.222879 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-kdq86" event={"ID":"117e290c-6ab5-4822-8212-5f637f523f3f","Type":"ContainerStarted","Data":"b8ef2ba7a65be85b95728147162ae701b9635c4bf543e26fd9b0fe14a2585991"} Jul 09 04:54:41.223034 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:41.222923 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-kdq86" Jul 09 04:54:41.655793 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:41.655764 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-p9f98_14521e76-b432-4e9f-95ee-9cb52fa8767c/volume-data-source-validator/0.log" Jul 09 04:54:42.658474 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:42.658438 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6prtw_675fded2-27b6-4bda-83ad-84f46e175548/dns/0.log" Jul 09 04:54:42.678828 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:42.678803 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6prtw_675fded2-27b6-4bda-83ad-84f46e175548/kube-rbac-proxy/0.log" Jul 09 04:54:42.796412 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:42.796350 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zpr9q_8104c5b9-011c-4e10-9a15-53bdd52870f0/dns-node-resolver/0.log" Jul 09 04:54:43.350911 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:43.350882 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dgz8s_0a7d54c6-9262-4462-8b4b-f35a678e2ddf/node-ca/0.log" Jul 09 04:54:44.402133 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:44.402102 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6wqjx_90f3ac8b-482f-439f-ae20-05262ae79c22/discovery/0.log" Jul 09 04:54:44.489214 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:44.489180 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-khl6s_136f41c8-89e3-4c37-9bf8-e4d68b96a781/istio-proxy/0.log" Jul 09 04:54:44.545885 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:44.545858 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-khl6s_136f41c8-89e3-4c37-9bf8-e4d68b96a781/istio-proxy/1.log" Jul 09 04:54:44.602456 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:44.602427 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-v9b9w_54b0094b-edbc-4be5-8c85-b6dbed9171cf/payload-pre-processing/0.log" Jul 09 04:54:44.672587 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:44.672509 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-prdjf_51c51e26-4ee8-4759-9885-2d7c389c662d/payload-processing/0.log" Jul 09 04:54:45.376917 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:45.376888 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ktvhk_3eb622e3-4ce7-4fe0-a2ae-b5c83a81fde6/serve-healthcheck-canary/0.log" Jul 09 04:54:45.838800 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:45.838756 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-8qmfm_900fd2e6-681b-451f-8f39-92d8d13bf9e5/insights-operator/0.log" Jul 09 04:54:45.839971 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:45.839943 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-8qmfm_900fd2e6-681b-451f-8f39-92d8d13bf9e5/insights-operator/1.log" Jul 09 04:54:45.860436 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:45.860419 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9hgjm_85856d16-c1e7-4990-9982-6e01d8ebc7f6/kube-rbac-proxy/0.log" Jul 09 04:54:45.879713 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:45.879698 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9hgjm_85856d16-c1e7-4990-9982-6e01d8ebc7f6/exporter/0.log" Jul 09 04:54:45.900347 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:45.900311 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9hgjm_85856d16-c1e7-4990-9982-6e01d8ebc7f6/extractor/0.log" Jul 09 04:54:47.240055 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:47.240027 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-kdq86" Jul 09 04:54:47.255933 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:47.255879 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-kdq86" podStartSLOduration=8.255864168 podStartE2EDuration="8.255864168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:54:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:54:41.236237833 +0000 UTC m=+2871.459856322" watchObservedRunningTime="2026-07-09 04:54:47.255864168 +0000 UTC m=+2877.479482656" Jul 09 04:54:48.243913 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:48.243875 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-f7cf9bf97-pmb2x_01c7c3c4-6115-404b-815b-058927f121a1/manager/0.log" Jul 09 04:54:48.486615 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:48.486580 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cff547d94-vs2gg_5a2d3035-3a6a-4db4-bc3f-de7a6c360ded/manager/0.log" Jul 09 04:54:48.504384 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:48.504276 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cff547d94-vs2gg_5a2d3035-3a6a-4db4-bc3f-de7a6c360ded/copy-manifests/0.log" Jul 09 04:54:49.980926 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:49.980894 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-645bc6d555-5fbz6_88bc6fa0-72c6-4142-a63c-22eae1310672/manager/0.log" Jul 09 04:54:50.450390 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:50.450360 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6wqjx_90f3ac8b-482f-439f-ae20-05262ae79c22/discovery/0.log" Jul 09 04:54:50.492132 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:50.492103 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-khl6s_136f41c8-89e3-4c37-9bf8-e4d68b96a781/istio-proxy/0.log" Jul 09 04:54:50.549564 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:50.549539 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-khl6s_136f41c8-89e3-4c37-9bf8-e4d68b96a781/istio-proxy/1.log" Jul 09 04:54:50.592879 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:50.592850 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-v9b9w_54b0094b-edbc-4be5-8c85-b6dbed9171cf/payload-pre-processing/0.log" Jul 09 04:54:50.650494 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:50.650468 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-prdjf_51c51e26-4ee8-4759-9885-2d7c389c662d/payload-processing/0.log" Jul 09 04:54:51.910599 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:51.910566 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6wqjx_90f3ac8b-482f-439f-ae20-05262ae79c22/discovery/0.log" Jul 09 04:54:51.955113 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:51.955077 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-khl6s_136f41c8-89e3-4c37-9bf8-e4d68b96a781/istio-proxy/0.log" Jul 09 04:54:52.010158 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:52.010131 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-khl6s_136f41c8-89e3-4c37-9bf8-e4d68b96a781/istio-proxy/1.log" Jul 09 04:54:52.059553 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:52.059477 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-v9b9w_54b0094b-edbc-4be5-8c85-b6dbed9171cf/payload-pre-processing/0.log" Jul 09 04:54:52.130517 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:52.130486 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-prdjf_51c51e26-4ee8-4759-9885-2d7c389c662d/payload-processing/0.log" Jul 09 04:54:54.737619 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:54.737588 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-rkghl_195e8f20-175e-4e62-a0f8-1635f6d29111/migrator/0.log" Jul 09 04:54:54.757276 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:54.757249 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-rkghl_195e8f20-175e-4e62-a0f8-1635f6d29111/graceful-termination/0.log" Jul 09 04:54:55.100256 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:55.100199 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-2lxfd_0c2522b9-d54d-4091-907e-eb613d77b5c9/kube-storage-version-migrator-operator/1.log" Jul 09 04:54:55.101714 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:55.101687 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-2lxfd_0c2522b9-d54d-4091-907e-eb613d77b5c9/kube-storage-version-migrator-operator/0.log" Jul 09 04:54:56.204707 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:56.204679 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k9th4_ee4f80f2-7676-489e-9fe0-99c52e384790/kube-multus-additional-cni-plugins/0.log" Jul 09 04:54:56.226795 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:56.226769 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k9th4_ee4f80f2-7676-489e-9fe0-99c52e384790/egress-router-binary-copy/0.log" Jul 09 04:54:56.247145 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:56.247109 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k9th4_ee4f80f2-7676-489e-9fe0-99c52e384790/cni-plugins/0.log" Jul 09 04:54:56.266527 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:56.266505 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k9th4_ee4f80f2-7676-489e-9fe0-99c52e384790/bond-cni-plugin/0.log" Jul 09 04:54:56.289883 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:56.289862 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k9th4_ee4f80f2-7676-489e-9fe0-99c52e384790/routeoverride-cni/0.log" Jul 09 04:54:56.309694 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:56.309672 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k9th4_ee4f80f2-7676-489e-9fe0-99c52e384790/whereabouts-cni-bincopy/0.log" Jul 09 04:54:56.331401 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:56.331374 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k9th4_ee4f80f2-7676-489e-9fe0-99c52e384790/whereabouts-cni/0.log" Jul 09 04:54:56.694610 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:56.694579 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-bv4mn_889db0de-781d-43d6-9df1-7a9eb15a1794/kube-multus/0.log" Jul 09 04:54:56.715918 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:56.715892 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-b4gcd_9fefcefc-4128-4422-98de-b17e5949cfc7/network-metrics-daemon/0.log" Jul 09 04:54:56.732903 ip-10-0-130-153 kubenswrapper[2534]: I0709 04:54:56.732876 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-b4gcd_9fefcefc-4128-4422-98de-b17e5949cfc7/kube-rbac-proxy/0.log"