Nov 04 03:46:23.962282 ip-10-0-129-134 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Nov 04 03:46:23.962295 ip-10-0-129-134 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Nov 04 03:46:23.962311 ip-10-0-129-134 systemd[1]: kubelet.service: Failed with result 'resources'. Nov 04 03:46:23.962567 ip-10-0-129-134 systemd[1]: Failed to start Kubernetes Kubelet. Nov 04 03:46:34.034056 ip-10-0-129-134 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Nov 04 03:46:34.034074 ip-10-0-129-134 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 40862e3de54b423096bed63a9aff4b15 -- Nov 04 03:48:47.524067 ip-10-0-129-134 systemd[1]: Starting Kubernetes Kubelet... Nov 04 03:48:47.880147 ip-10-0-129-134 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 03:48:47.880147 ip-10-0-129-134 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Nov 04 03:48:47.880147 ip-10-0-129-134 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 03:48:47.880147 ip-10-0-129-134 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 03:48:47.888335 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888194 2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 03:48:47.888335 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888330 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 03:48:47.888335 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888334 2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 03:48:47.888335 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888337 2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 03:48:47.888335 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888341 2575 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 03:48:47.888335 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888344 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 03:48:47.888335 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888347 2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 03:48:47.888335 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888350 2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 03:48:47.888606 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888352 2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 03:48:47.888606 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888355 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 03:48:47.888606 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888358 2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 03:48:47.888606 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888361 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 03:48:47.888606 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888364 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 03:48:47.888606 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888366 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 03:48:47.888606 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888369 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 03:48:47.888606 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888371 2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 03:48:47.888606 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888374 2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 03:48:47.888606 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888377 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 03:48:47.888606 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888380 2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 03:48:47.888606 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888382 2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 03:48:47.888606 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888385 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 03:48:47.888606 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888388 2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 03:48:47.888606 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888391 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 03:48:47.888606 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888393 2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 03:48:47.888606 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888396 2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 03:48:47.888606 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888399 2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 03:48:47.888606 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888401 2575 feature_gate.go:386] unrecognized feature gate: Example Nov 04 03:48:47.889072 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888404 2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 03:48:47.889072 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888406 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 03:48:47.889072 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888409 2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 03:48:47.889072 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888411 2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 03:48:47.889072 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888414 2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 03:48:47.889072 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888416 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 03:48:47.889072 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888419 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 03:48:47.889072 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888421 2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 03:48:47.889072 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888425 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 03:48:47.889072 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888427 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 03:48:47.889072 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888430 2575 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 03:48:47.889072 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888433 2575 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 03:48:47.889072 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888435 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 03:48:47.889072 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888438 2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 03:48:47.889072 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888441 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 03:48:47.889072 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888443 2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 03:48:47.889072 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888448 2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 03:48:47.889072 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888453 2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 03:48:47.889072 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888455 2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 03:48:47.889072 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888458 2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 03:48:47.889537 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888461 2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 03:48:47.889537 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888464 2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 03:48:47.889537 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888467 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 03:48:47.889537 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888470 2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 03:48:47.889537 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888473 2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 03:48:47.889537 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888475 2575 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 03:48:47.889537 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888478 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 03:48:47.889537 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888481 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 03:48:47.889537 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888483 2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 03:48:47.889537 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888486 2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 03:48:47.889537 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888488 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 03:48:47.889537 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888491 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 03:48:47.889537 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888494 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 03:48:47.889537 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888496 2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 03:48:47.889537 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888499 2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 03:48:47.889537 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888502 2575 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 03:48:47.889537 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888506 2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 03:48:47.889537 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888510 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 03:48:47.889537 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888513 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 03:48:47.889993 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888516 2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 03:48:47.889993 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888520 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 03:48:47.889993 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888523 2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 03:48:47.889993 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888526 2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 03:48:47.889993 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888528 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 03:48:47.889993 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888531 2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 03:48:47.889993 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888534 2575 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 03:48:47.889993 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888537 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 03:48:47.889993 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888542 2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 03:48:47.889993 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888545 2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 03:48:47.889993 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888548 2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 03:48:47.889993 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888551 2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 03:48:47.889993 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888554 2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 03:48:47.889993 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888557 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 03:48:47.889993 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888559 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 03:48:47.889993 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888562 2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 03:48:47.889993 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888564 2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 03:48:47.889993 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888568 2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 03:48:47.889993 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888571 2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 03:48:47.889993 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888574 2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 03:48:47.890460 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888577 2575 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 03:48:47.890460 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888586 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 03:48:47.890460 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888589 2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 03:48:47.890460 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888592 2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 03:48:47.890460 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888595 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 03:48:47.890460 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888597 2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 03:48:47.890460 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888600 2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 03:48:47.890460 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888603 2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 03:48:47.890460 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888605 2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 03:48:47.890460 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888608 2575 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 03:48:47.890460 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888611 2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 03:48:47.890460 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888614 2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 03:48:47.890460 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888617 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 03:48:47.890460 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888619 2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 03:48:47.890460 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888622 2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 03:48:47.890460 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888627 2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 03:48:47.890460 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888629 2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 03:48:47.890460 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888632 2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 03:48:47.890460 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888635 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 03:48:47.890460 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.888637 2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 03:48:47.890944 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889523 2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 03:48:47.890944 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889533 2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 03:48:47.890944 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889537 2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 03:48:47.890944 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889541 2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 03:48:47.890944 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889544 2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 03:48:47.890944 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889547 2575 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 03:48:47.890944 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889550 2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 03:48:47.890944 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889553 2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 03:48:47.890944 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889555 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 03:48:47.890944 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889558 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 03:48:47.890944 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889562 2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 03:48:47.890944 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889565 2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 03:48:47.890944 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889568 2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 03:48:47.890944 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889571 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 03:48:47.890944 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889574 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 03:48:47.890944 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889576 2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 03:48:47.890944 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889579 2575 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 03:48:47.890944 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889581 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 03:48:47.890944 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889584 2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 03:48:47.891400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889587 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 03:48:47.891400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889589 2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 03:48:47.891400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889592 2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 03:48:47.891400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889595 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 03:48:47.891400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889597 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 03:48:47.891400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889600 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 03:48:47.891400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889603 2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 03:48:47.891400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889605 2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 03:48:47.891400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889608 2575 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 03:48:47.891400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889611 2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 03:48:47.891400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889614 2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 03:48:47.891400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889617 2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 03:48:47.891400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889619 2575 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 03:48:47.891400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889622 2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 03:48:47.891400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889625 2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 03:48:47.891400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889628 2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 03:48:47.891400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889631 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 03:48:47.891400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889633 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 03:48:47.891400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889636 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 03:48:47.891400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889638 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 03:48:47.891892 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889641 2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 03:48:47.891892 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889643 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 03:48:47.891892 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889646 2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 03:48:47.891892 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889648 2575 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 03:48:47.891892 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889652 2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 03:48:47.891892 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889654 2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 03:48:47.891892 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889657 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 03:48:47.891892 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889660 2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 03:48:47.891892 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889663 2575 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 03:48:47.891892 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889665 2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 03:48:47.891892 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889668 2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 03:48:47.891892 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889670 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 03:48:47.891892 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889673 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 03:48:47.891892 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889676 2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 03:48:47.891892 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889678 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 03:48:47.891892 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889681 2575 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 03:48:47.891892 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889683 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 03:48:47.891892 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889686 2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 03:48:47.891892 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889688 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 03:48:47.892423 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889691 2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 03:48:47.892423 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889693 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 03:48:47.892423 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889697 2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 03:48:47.892423 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889702 2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 03:48:47.892423 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889704 2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 03:48:47.892423 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889707 2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 03:48:47.892423 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889709 2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 03:48:47.892423 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889714 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 03:48:47.892423 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889717 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 03:48:47.892423 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889720 2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 03:48:47.892423 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889722 2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 03:48:47.892423 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889725 2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 03:48:47.892423 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889727 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 03:48:47.892423 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889730 2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 03:48:47.892423 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889733 2575 feature_gate.go:386] unrecognized feature gate: Example Nov 04 03:48:47.892423 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889736 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 03:48:47.892423 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889738 2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 03:48:47.892423 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889741 2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 03:48:47.892423 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889744 2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 03:48:47.892897 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889747 2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 03:48:47.892897 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889750 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 03:48:47.892897 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889768 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 03:48:47.892897 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889772 2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 03:48:47.892897 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889776 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 03:48:47.892897 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889780 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 03:48:47.892897 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889784 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 03:48:47.892897 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889787 2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 03:48:47.892897 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889790 2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 03:48:47.892897 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889793 2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 03:48:47.892897 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889796 2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 03:48:47.892897 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889800 2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 03:48:47.892897 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889803 2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 03:48:47.892897 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889805 2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 03:48:47.892897 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889808 2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 03:48:47.892897 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889811 2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 03:48:47.892897 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889813 2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 03:48:47.892897 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889816 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 03:48:47.892897 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889818 2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 03:48:47.892897 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889821 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 03:48:47.893379 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889824 2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 03:48:47.893379 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889827 2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 03:48:47.893379 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889829 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 03:48:47.893379 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889831 2575 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 03:48:47.893379 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889834 2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 03:48:47.893379 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889837 2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 03:48:47.893379 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889839 2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 03:48:47.893379 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889842 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 03:48:47.893379 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.889844 2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 03:48:47.893379 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890396 2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 03:48:47.893379 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890402 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 03:48:47.893379 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890406 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 03:48:47.893379 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890409 2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 03:48:47.893379 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890412 2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 03:48:47.893379 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890414 2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 03:48:47.893379 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890417 2575 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 03:48:47.893379 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890420 2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 03:48:47.893379 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890422 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 03:48:47.893379 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890425 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 03:48:47.893855 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890427 2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 03:48:47.893855 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890430 2575 feature_gate.go:386] unrecognized feature gate: Example Nov 04 03:48:47.893855 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890433 2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 03:48:47.893855 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890436 2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 03:48:47.893855 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890438 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 03:48:47.893855 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890441 2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 03:48:47.893855 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890443 2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 03:48:47.893855 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890446 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 03:48:47.893855 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890448 2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 03:48:47.893855 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890451 2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 03:48:47.893855 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890454 2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 03:48:47.893855 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890457 2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 03:48:47.893855 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890459 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 03:48:47.893855 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890462 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 03:48:47.893855 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890467 2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 03:48:47.893855 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890471 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 03:48:47.893855 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890474 2575 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 03:48:47.893855 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890477 2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 03:48:47.893855 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890480 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 03:48:47.893855 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890483 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 03:48:47.894337 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890486 2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 03:48:47.894337 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890488 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 03:48:47.894337 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890491 2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 03:48:47.894337 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890494 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 03:48:47.894337 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890496 2575 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 03:48:47.894337 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890500 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 03:48:47.894337 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890502 2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 03:48:47.894337 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890505 2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 03:48:47.894337 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890508 2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 03:48:47.894337 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890512 2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 03:48:47.894337 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890516 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 03:48:47.894337 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890519 2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 03:48:47.894337 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890522 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 03:48:47.894337 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890524 2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 03:48:47.894337 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890527 2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 03:48:47.894337 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890529 2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 03:48:47.894337 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890532 2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 03:48:47.894337 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890534 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 03:48:47.894337 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890537 2575 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 03:48:47.894794 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890540 2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 03:48:47.894794 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890543 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 03:48:47.894794 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890546 2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 03:48:47.894794 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890549 2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 03:48:47.894794 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890552 2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 03:48:47.894794 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890554 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 03:48:47.894794 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890557 2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 03:48:47.894794 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890559 2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 03:48:47.894794 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890562 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 03:48:47.894794 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890564 2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 03:48:47.894794 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890567 2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 03:48:47.894794 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890571 2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 03:48:47.894794 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890573 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 03:48:47.894794 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890576 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 03:48:47.894794 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890578 2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 03:48:47.894794 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890581 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 03:48:47.894794 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890583 2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 03:48:47.894794 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890586 2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 03:48:47.894794 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890588 2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 03:48:47.894794 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890592 2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 03:48:47.895274 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890594 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 03:48:47.895274 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890597 2575 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 03:48:47.895274 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890599 2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 03:48:47.895274 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890602 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 03:48:47.895274 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890604 2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 03:48:47.895274 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890607 2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 03:48:47.895274 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890609 2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 03:48:47.895274 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890612 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 03:48:47.895274 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890614 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 03:48:47.895274 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890617 2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 03:48:47.895274 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890619 2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 03:48:47.895274 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890621 2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 03:48:47.895274 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890624 2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 03:48:47.895274 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890626 2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 03:48:47.895274 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890629 2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 03:48:47.895274 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890631 2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 03:48:47.895274 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890634 2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 03:48:47.895274 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890637 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 03:48:47.895274 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890642 2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 03:48:47.895274 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890644 2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 03:48:47.895772 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890647 2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 03:48:47.895772 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890649 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 03:48:47.895772 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890652 2575 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 03:48:47.895772 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890655 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 03:48:47.895772 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890658 2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 03:48:47.895772 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890661 2575 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 03:48:47.895772 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890663 2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 03:48:47.895772 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890666 2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 03:48:47.895772 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890669 2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 03:48:47.895772 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890672 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 03:48:47.895772 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890674 2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 03:48:47.895772 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890677 2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 03:48:47.895772 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890680 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 03:48:47.895772 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890683 2575 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 03:48:47.895772 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890686 2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 03:48:47.895772 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890688 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 03:48:47.895772 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.890691 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 03:48:47.895772 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891856 2575 flags.go:64] FLAG: --address="0.0.0.0" Nov 04 03:48:47.895772 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891868 2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Nov 04 03:48:47.895772 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891876 2575 flags.go:64] FLAG: --anonymous-auth="true" Nov 04 03:48:47.895772 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891881 2575 flags.go:64] FLAG: --application-metrics-count-limit="100" Nov 04 03:48:47.896279 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891886 2575 flags.go:64] FLAG: --authentication-token-webhook="false" Nov 04 03:48:47.896279 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891889 2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Nov 04 03:48:47.896279 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891894 2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Nov 04 03:48:47.896279 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891898 2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Nov 04 03:48:47.896279 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891902 2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Nov 04 03:48:47.896279 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891905 2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Nov 04 03:48:47.896279 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891909 2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Nov 04 03:48:47.896279 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891912 2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Nov 04 03:48:47.896279 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891916 2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Nov 04 03:48:47.896279 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891919 2575 flags.go:64] FLAG: --cgroup-root="" Nov 04 03:48:47.896279 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891923 2575 flags.go:64] FLAG: --cgroups-per-qos="true" Nov 04 03:48:47.896279 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891925 2575 flags.go:64] FLAG: --client-ca-file="" Nov 04 03:48:47.896279 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891929 2575 flags.go:64] FLAG: --cloud-provider="external" Nov 04 03:48:47.896279 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891932 2575 flags.go:64] FLAG: --cluster-dns="[]" Nov 04 03:48:47.896279 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891936 2575 flags.go:64] FLAG: --cluster-domain="" Nov 04 03:48:47.896279 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891939 2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Nov 04 03:48:47.896279 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891942 2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Nov 04 03:48:47.896279 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891945 2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Nov 04 03:48:47.896279 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891949 2575 flags.go:64] FLAG: --container-log-max-files="5" Nov 04 03:48:47.896279 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891953 2575 flags.go:64] FLAG: --container-log-max-size="10Mi" Nov 04 03:48:47.896279 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891956 2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Nov 04 03:48:47.896279 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891959 2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Nov 04 03:48:47.896279 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891963 2575 flags.go:64] FLAG: --containerd-namespace="k8s.io" Nov 04 03:48:47.896887 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891966 2575 flags.go:64] FLAG: --contention-profiling="false" Nov 04 03:48:47.896887 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891970 2575 flags.go:64] FLAG: --cpu-cfs-quota="true" Nov 04 03:48:47.896887 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891973 2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Nov 04 03:48:47.896887 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891976 2575 flags.go:64] FLAG: --cpu-manager-policy="none" Nov 04 03:48:47.896887 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891979 2575 flags.go:64] FLAG: --cpu-manager-policy-options="" Nov 04 03:48:47.896887 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891984 2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Nov 04 03:48:47.896887 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891987 2575 flags.go:64] FLAG: --enable-controller-attach-detach="true" Nov 04 03:48:47.896887 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891990 2575 flags.go:64] FLAG: --enable-debugging-handlers="true" Nov 04 03:48:47.896887 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891992 2575 flags.go:64] FLAG: --enable-load-reader="false" Nov 04 03:48:47.896887 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891995 2575 flags.go:64] FLAG: --enable-server="true" Nov 04 03:48:47.896887 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.891999 2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Nov 04 03:48:47.896887 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892005 2575 flags.go:64] FLAG: --event-burst="100" Nov 04 03:48:47.896887 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892008 2575 flags.go:64] FLAG: --event-qps="50" Nov 04 03:48:47.896887 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892011 2575 flags.go:64] FLAG: --event-storage-age-limit="default=0" Nov 04 03:48:47.896887 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892014 2575 flags.go:64] FLAG: --event-storage-event-limit="default=0" Nov 04 03:48:47.896887 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892017 2575 flags.go:64] FLAG: --eviction-hard="" Nov 04 03:48:47.896887 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892027 2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Nov 04 03:48:47.896887 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892030 2575 flags.go:64] FLAG: --eviction-minimum-reclaim="" Nov 04 03:48:47.896887 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892033 2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Nov 04 03:48:47.896887 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892036 2575 flags.go:64] FLAG: --eviction-soft="" Nov 04 03:48:47.896887 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892040 2575 flags.go:64] FLAG: --eviction-soft-grace-period="" Nov 04 03:48:47.896887 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892043 2575 flags.go:64] FLAG: --exit-on-lock-contention="false" Nov 04 03:48:47.896887 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892046 2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Nov 04 03:48:47.896887 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892049 2575 flags.go:64] FLAG: --experimental-mounter-path="" Nov 04 03:48:47.896887 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892052 2575 flags.go:64] FLAG: --fail-cgroupv1="true" Nov 04 03:48:47.897512 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892055 2575 flags.go:64] FLAG: --fail-swap-on="true" Nov 04 03:48:47.897512 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892058 2575 flags.go:64] FLAG: --feature-gates="" Nov 04 03:48:47.897512 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892062 2575 flags.go:64] FLAG: --file-check-frequency="20s" Nov 04 03:48:47.897512 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892065 2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Nov 04 03:48:47.897512 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892069 2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Nov 04 03:48:47.897512 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892072 2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Nov 04 03:48:47.897512 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892075 2575 flags.go:64] FLAG: --healthz-port="10248" Nov 04 03:48:47.897512 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892078 2575 flags.go:64] FLAG: --help="false" Nov 04 03:48:47.897512 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892081 2575 flags.go:64] FLAG: --hostname-override="ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:48:47.897512 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892086 2575 flags.go:64] FLAG: --housekeeping-interval="10s" Nov 04 03:48:47.897512 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892089 2575 flags.go:64] FLAG: --http-check-frequency="20s" Nov 04 03:48:47.897512 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892092 2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Nov 04 03:48:47.897512 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892096 2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Nov 04 03:48:47.897512 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892099 2575 flags.go:64] FLAG: --image-gc-high-threshold="85" Nov 04 03:48:47.897512 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892102 2575 flags.go:64] FLAG: --image-gc-low-threshold="80" Nov 04 03:48:47.897512 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892105 2575 flags.go:64] FLAG: --image-service-endpoint="" Nov 04 03:48:47.897512 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892108 2575 flags.go:64] FLAG: --kernel-memcg-notification="false" Nov 04 03:48:47.897512 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892110 2575 flags.go:64] FLAG: --kube-api-burst="100" Nov 04 03:48:47.897512 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892113 2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Nov 04 03:48:47.897512 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892116 2575 flags.go:64] FLAG: --kube-api-qps="50" Nov 04 03:48:47.897512 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892119 2575 flags.go:64] FLAG: --kube-reserved="" Nov 04 03:48:47.897512 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892122 2575 flags.go:64] FLAG: --kube-reserved-cgroup="" Nov 04 03:48:47.897512 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892125 2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Nov 04 03:48:47.897512 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892128 2575 flags.go:64] FLAG: --kubelet-cgroups="" Nov 04 03:48:47.898158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892131 2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Nov 04 03:48:47.898158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892134 2575 flags.go:64] FLAG: --lock-file="" Nov 04 03:48:47.898158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892137 2575 flags.go:64] FLAG: --log-cadvisor-usage="false" Nov 04 03:48:47.898158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892140 2575 flags.go:64] FLAG: --log-flush-frequency="5s" Nov 04 03:48:47.898158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892143 2575 flags.go:64] FLAG: --log-json-info-buffer-size="0" Nov 04 03:48:47.898158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892149 2575 flags.go:64] FLAG: --log-json-split-stream="false" Nov 04 03:48:47.898158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892154 2575 flags.go:64] FLAG: --log-text-info-buffer-size="0" Nov 04 03:48:47.898158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892157 2575 flags.go:64] FLAG: --log-text-split-stream="false" Nov 04 03:48:47.898158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892160 2575 flags.go:64] FLAG: --logging-format="text" Nov 04 03:48:47.898158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892163 2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Nov 04 03:48:47.898158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892167 2575 flags.go:64] FLAG: --make-iptables-util-chains="true" Nov 04 03:48:47.898158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892170 2575 flags.go:64] FLAG: --manifest-url="" Nov 04 03:48:47.898158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892173 2575 flags.go:64] FLAG: --manifest-url-header="" Nov 04 03:48:47.898158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892178 2575 flags.go:64] FLAG: --max-housekeeping-interval="15s" Nov 04 03:48:47.898158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892181 2575 flags.go:64] FLAG: --max-open-files="1000000" Nov 04 03:48:47.898158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892185 2575 flags.go:64] FLAG: --max-pods="110" Nov 04 03:48:47.898158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892188 2575 flags.go:64] FLAG: --maximum-dead-containers="-1" Nov 04 03:48:47.898158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892191 2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Nov 04 03:48:47.898158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892195 2575 flags.go:64] FLAG: --memory-manager-policy="None" Nov 04 03:48:47.898158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892198 2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Nov 04 03:48:47.898158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892201 2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Nov 04 03:48:47.898158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892204 2575 flags.go:64] FLAG: --node-ip="0.0.0.0" Nov 04 03:48:47.898158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892207 2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Nov 04 03:48:47.898158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892216 2575 flags.go:64] FLAG: --node-status-max-images="50" Nov 04 03:48:47.898158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892219 2575 flags.go:64] FLAG: --node-status-update-frequency="10s" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892222 2575 flags.go:64] FLAG: --oom-score-adj="-999" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892225 2575 flags.go:64] FLAG: --pod-cidr="" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892228 2575 flags.go:64] FLAG: --pod-manifest-path="" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892231 2575 flags.go:64] FLAG: --pod-max-pids="-1" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892235 2575 flags.go:64] FLAG: --pods-per-core="0" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892237 2575 flags.go:64] FLAG: --port="10250" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892241 2575 flags.go:64] FLAG: --protect-kernel-defaults="false" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892243 2575 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-09942282de6148589" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892246 2575 flags.go:64] FLAG: --qos-reserved="" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892250 2575 flags.go:64] FLAG: --read-only-port="10255" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892253 2575 flags.go:64] FLAG: --register-node="true" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892256 2575 flags.go:64] FLAG: --register-with-taints="" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892259 2575 flags.go:64] FLAG: --registry-burst="10" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892265 2575 flags.go:64] FLAG: --registry-qps="5" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892268 2575 flags.go:64] FLAG: --reserved-cpus="" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892271 2575 flags.go:64] FLAG: --reserved-memory="" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892275 2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892278 2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892282 2575 flags.go:64] FLAG: --rotate-certificates="false" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892285 2575 flags.go:64] FLAG: --rotate-server-certificates="false" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892288 2575 flags.go:64] FLAG: --runonce="false" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892291 2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892294 2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892297 2575 flags.go:64] FLAG: --seccomp-default="false" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892300 2575 flags.go:64] FLAG: --serialize-image-pulls="true" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892303 2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Nov 04 03:48:47.898733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892307 2575 flags.go:64] FLAG: --storage-driver-db="cadvisor" Nov 04 03:48:47.899375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892311 2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Nov 04 03:48:47.899375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892314 2575 flags.go:64] FLAG: --storage-driver-password="root" Nov 04 03:48:47.899375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892317 2575 flags.go:64] FLAG: --storage-driver-secure="false" Nov 04 03:48:47.899375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892320 2575 flags.go:64] FLAG: --storage-driver-table="stats" Nov 04 03:48:47.899375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892323 2575 flags.go:64] FLAG: --storage-driver-user="root" Nov 04 03:48:47.899375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892326 2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Nov 04 03:48:47.899375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892329 2575 flags.go:64] FLAG: --sync-frequency="1m0s" Nov 04 03:48:47.899375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892332 2575 flags.go:64] FLAG: --system-cgroups="" Nov 04 03:48:47.899375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892335 2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Nov 04 03:48:47.899375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892341 2575 flags.go:64] FLAG: --system-reserved-cgroup="" Nov 04 03:48:47.899375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892344 2575 flags.go:64] FLAG: --tls-cert-file="" Nov 04 03:48:47.899375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892347 2575 flags.go:64] FLAG: --tls-cipher-suites="[]" Nov 04 03:48:47.899375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892352 2575 flags.go:64] FLAG: --tls-min-version="" Nov 04 03:48:47.899375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892355 2575 flags.go:64] FLAG: --tls-private-key-file="" Nov 04 03:48:47.899375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892358 2575 flags.go:64] FLAG: --topology-manager-policy="none" Nov 04 03:48:47.899375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892361 2575 flags.go:64] FLAG: --topology-manager-policy-options="" Nov 04 03:48:47.899375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892364 2575 flags.go:64] FLAG: --topology-manager-scope="container" Nov 04 03:48:47.899375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892367 2575 flags.go:64] FLAG: --v="2" Nov 04 03:48:47.899375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892373 2575 flags.go:64] FLAG: --version="false" Nov 04 03:48:47.899375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892378 2575 flags.go:64] FLAG: --vmodule="" Nov 04 03:48:47.899375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892382 2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Nov 04 03:48:47.899375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892386 2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Nov 04 03:48:47.899375 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892526 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 03:48:47.899375 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892531 2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 03:48:47.899973 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892535 2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 03:48:47.899973 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892538 2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 03:48:47.899973 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892541 2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 03:48:47.899973 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892544 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 03:48:47.899973 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892546 2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 03:48:47.899973 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892549 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 03:48:47.899973 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892552 2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 03:48:47.899973 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892554 2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 03:48:47.899973 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892558 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 03:48:47.899973 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892561 2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 03:48:47.899973 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892564 2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 03:48:47.899973 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892566 2575 feature_gate.go:386] unrecognized feature gate: Example Nov 04 03:48:47.899973 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892569 2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 03:48:47.899973 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892572 2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 03:48:47.899973 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892574 2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 03:48:47.899973 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892577 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 03:48:47.899973 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892580 2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 03:48:47.899973 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892583 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 03:48:47.899973 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892585 2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 03:48:47.900469 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892588 2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 03:48:47.900469 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892591 2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 03:48:47.900469 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892593 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 03:48:47.900469 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892596 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 03:48:47.900469 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892598 2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 03:48:47.900469 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892601 2575 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 03:48:47.900469 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892604 2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 03:48:47.900469 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892608 2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 03:48:47.900469 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892610 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 03:48:47.900469 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892613 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 03:48:47.900469 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892615 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 03:48:47.900469 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892618 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 03:48:47.900469 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892622 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 03:48:47.900469 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892624 2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 03:48:47.900469 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892627 2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 03:48:47.900469 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892630 2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 03:48:47.900469 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892632 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 03:48:47.900469 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892635 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 03:48:47.900469 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892638 2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 03:48:47.900957 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892640 2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 03:48:47.900957 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892643 2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 03:48:47.900957 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892646 2575 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 03:48:47.900957 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892649 2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 03:48:47.900957 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892652 2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 03:48:47.900957 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892654 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 03:48:47.900957 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892657 2575 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 03:48:47.900957 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892660 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 03:48:47.900957 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892663 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 03:48:47.900957 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892665 2575 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 03:48:47.900957 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892668 2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 03:48:47.900957 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892671 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 03:48:47.900957 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892673 2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 03:48:47.900957 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892676 2575 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 03:48:47.900957 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892678 2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 03:48:47.900957 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892681 2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 03:48:47.900957 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892683 2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 03:48:47.900957 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892686 2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 03:48:47.900957 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892689 2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 03:48:47.900957 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892691 2575 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 03:48:47.901462 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892697 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 03:48:47.901462 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892699 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 03:48:47.901462 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892702 2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 03:48:47.901462 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892704 2575 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 03:48:47.901462 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892707 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 03:48:47.901462 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892711 2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 03:48:47.901462 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892713 2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 03:48:47.901462 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892716 2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 03:48:47.901462 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892718 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 03:48:47.901462 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892722 2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 03:48:47.901462 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892726 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 03:48:47.901462 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892729 2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 03:48:47.901462 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892732 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 03:48:47.901462 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892735 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 03:48:47.901462 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892738 2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 03:48:47.901462 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892740 2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 03:48:47.901462 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892751 2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 03:48:47.901462 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892765 2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 03:48:47.901462 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892768 2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 03:48:47.902020 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892771 2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 03:48:47.902020 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892774 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 03:48:47.902020 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892777 2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 03:48:47.902020 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892780 2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 03:48:47.902020 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892782 2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 03:48:47.902020 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892785 2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 03:48:47.902020 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892788 2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 03:48:47.902020 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892790 2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 03:48:47.902020 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892793 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 03:48:47.902020 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892796 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 03:48:47.902020 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892798 2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 03:48:47.902020 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892801 2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 03:48:47.902020 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892804 2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 03:48:47.902020 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892808 2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 03:48:47.902020 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892811 2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 03:48:47.902020 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892814 2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 03:48:47.902020 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892816 2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 03:48:47.902020 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892819 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 03:48:47.902020 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892825 2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 03:48:47.902459 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892829 2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 03:48:47.902459 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892832 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 03:48:47.902459 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892835 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 03:48:47.902459 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892837 2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 03:48:47.902459 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892840 2575 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 03:48:47.902459 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892843 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 03:48:47.902459 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892846 2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 03:48:47.902459 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892848 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 03:48:47.902459 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.892874 2575 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 03:48:47.902459 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892913 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 03:48:47.902459 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892918 2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 03:48:47.902459 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892927 2575 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 03:48:47.902459 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892930 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 03:48:47.902459 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892933 2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 03:48:47.902459 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892935 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 03:48:47.902459 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892938 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 03:48:47.902865 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892940 2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 03:48:47.902865 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892943 2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 03:48:47.902865 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892946 2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 03:48:47.902865 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892949 2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 03:48:47.902865 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892951 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 03:48:47.902865 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892954 2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 03:48:47.902865 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892957 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 03:48:47.902865 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892959 2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 03:48:47.902865 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892962 2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 03:48:47.902865 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892965 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 03:48:47.902865 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892967 2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 03:48:47.902865 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892972 2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 03:48:47.902865 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892975 2575 feature_gate.go:386] unrecognized feature gate: Example Nov 04 03:48:47.902865 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892977 2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 03:48:47.902865 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892980 2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 03:48:47.902865 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892984 2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 03:48:47.902865 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892986 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 03:48:47.902865 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892989 2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 03:48:47.902865 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892992 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 03:48:47.903309 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892994 2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 03:48:47.903309 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892997 2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 03:48:47.903309 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.892999 2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 03:48:47.903309 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893002 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 03:48:47.903309 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893005 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 03:48:47.903309 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893008 2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 03:48:47.903309 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893010 2575 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 03:48:47.903309 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893013 2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 03:48:47.903309 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893015 2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 03:48:47.903309 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893020 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 03:48:47.903309 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893022 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 03:48:47.903309 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893025 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 03:48:47.903309 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893028 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 03:48:47.903309 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893030 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 03:48:47.903309 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893033 2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 03:48:47.903309 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893036 2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 03:48:47.903309 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893038 2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 03:48:47.903309 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893041 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 03:48:47.903309 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893043 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 03:48:47.903766 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893046 2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 03:48:47.903766 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893048 2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 03:48:47.903766 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893051 2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 03:48:47.903766 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893054 2575 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 03:48:47.903766 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893057 2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 03:48:47.903766 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893061 2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 03:48:47.903766 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893064 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 03:48:47.903766 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893066 2575 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 03:48:47.903766 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893069 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 03:48:47.903766 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893073 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 03:48:47.903766 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893076 2575 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 03:48:47.903766 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893079 2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 03:48:47.903766 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893082 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 03:48:47.903766 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893085 2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 03:48:47.903766 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893087 2575 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 03:48:47.903766 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893090 2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 03:48:47.903766 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893092 2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 03:48:47.903766 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893095 2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 03:48:47.903766 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893097 2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 03:48:47.903766 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893100 2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 03:48:47.904267 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893103 2575 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 03:48:47.904267 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893105 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 03:48:47.904267 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893108 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 03:48:47.904267 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893111 2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 03:48:47.904267 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893114 2575 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 03:48:47.904267 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893116 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 03:48:47.904267 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893119 2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 03:48:47.904267 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893121 2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 03:48:47.904267 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893124 2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 03:48:47.904267 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893126 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 03:48:47.904267 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893131 2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 03:48:47.904267 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893134 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 03:48:47.904267 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893137 2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 03:48:47.904267 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893140 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 03:48:47.904267 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893142 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 03:48:47.904267 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893145 2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 03:48:47.904267 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893147 2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 03:48:47.904267 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893151 2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 03:48:47.904267 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893154 2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 03:48:47.904267 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893156 2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 03:48:47.904730 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893159 2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 03:48:47.904730 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893163 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 03:48:47.904730 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893165 2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 03:48:47.904730 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893168 2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 03:48:47.904730 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893171 2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 03:48:47.904730 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893173 2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 03:48:47.904730 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893176 2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 03:48:47.904730 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893178 2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 03:48:47.904730 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893181 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 03:48:47.904730 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893184 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 03:48:47.904730 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893186 2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 03:48:47.904730 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893189 2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 03:48:47.904730 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893191 2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 03:48:47.904730 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893194 2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 03:48:47.904730 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893197 2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 03:48:47.904730 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893199 2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 03:48:47.904730 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893203 2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 03:48:47.904730 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893205 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 03:48:47.904730 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893210 2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 03:48:47.905207 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893214 2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 03:48:47.905207 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.893217 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 03:48:47.905207 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.900437 2575 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Nov 04 03:48:47.905207 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.900459 2575 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Nov 04 03:48:47.905207 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.901105 2575 watchdog_linux.go:95] "Systemd watchdog is not enabled" Nov 04 03:48:47.905207 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.901111 2575 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Nov 04 03:48:47.905207 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901171 2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 03:48:47.905207 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901177 2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 03:48:47.905207 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901181 2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 03:48:47.905207 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901184 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 03:48:47.905207 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901187 2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 03:48:47.905207 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901189 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 03:48:47.905207 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901192 2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 03:48:47.905207 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901195 2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 03:48:47.905207 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901197 2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 03:48:47.905207 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901200 2575 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 03:48:47.905207 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901203 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 03:48:47.905207 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901205 2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 03:48:47.905207 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901208 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 03:48:47.905207 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901212 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 03:48:47.905680 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901214 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 03:48:47.905680 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901217 2575 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 03:48:47.905680 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901219 2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 03:48:47.905680 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901222 2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 03:48:47.905680 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901224 2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 03:48:47.905680 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901227 2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 03:48:47.905680 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901229 2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 03:48:47.905680 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901232 2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 03:48:47.905680 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901235 2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 03:48:47.905680 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901237 2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 03:48:47.905680 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901240 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 03:48:47.905680 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901243 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 03:48:47.905680 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901246 2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 03:48:47.905680 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901248 2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 03:48:47.905680 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901252 2575 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 03:48:47.905680 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901256 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 03:48:47.905680 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901259 2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 03:48:47.905680 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901262 2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 03:48:47.905680 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901265 2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 03:48:47.905680 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901267 2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 03:48:47.906266 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901270 2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 03:48:47.906266 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901273 2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 03:48:47.906266 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901277 2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 03:48:47.906266 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901288 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 03:48:47.906266 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901291 2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 03:48:47.906266 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901294 2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 03:48:47.906266 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901297 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 03:48:47.906266 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901300 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 03:48:47.906266 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901302 2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 03:48:47.906266 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901305 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 03:48:47.906266 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901308 2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 03:48:47.906266 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901311 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 03:48:47.906266 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901313 2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 03:48:47.906266 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901316 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 03:48:47.906266 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901318 2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 03:48:47.906266 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901321 2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 03:48:47.906266 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901324 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 03:48:47.906266 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901326 2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 03:48:47.906684 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901329 2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 03:48:47.906684 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901332 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 03:48:47.906684 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901335 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 03:48:47.906684 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901337 2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 03:48:47.906684 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901340 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 03:48:47.906684 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901342 2575 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 03:48:47.906684 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901345 2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 03:48:47.906684 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901348 2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 03:48:47.906684 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901351 2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 03:48:47.906684 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901354 2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 03:48:47.906684 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901357 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 03:48:47.906684 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901360 2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 03:48:47.906684 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901364 2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 03:48:47.906684 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901366 2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 03:48:47.906684 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901369 2575 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 03:48:47.906684 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901374 2575 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 03:48:47.906684 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901376 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 03:48:47.906684 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901379 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 03:48:47.906684 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901382 2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 03:48:47.906684 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901386 2575 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 03:48:47.907172 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901389 2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 03:48:47.907172 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901392 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 03:48:47.907172 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901394 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 03:48:47.907172 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901397 2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 03:48:47.907172 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901399 2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 03:48:47.907172 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901402 2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 03:48:47.907172 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901405 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 03:48:47.907172 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901407 2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 03:48:47.907172 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901410 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 03:48:47.907172 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901413 2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 03:48:47.907172 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901415 2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 03:48:47.907172 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901418 2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 03:48:47.907172 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901421 2575 feature_gate.go:386] unrecognized feature gate: Example Nov 04 03:48:47.907172 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901424 2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 03:48:47.907172 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901427 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 03:48:47.907172 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901431 2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 03:48:47.907172 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901434 2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 03:48:47.907172 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901437 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 03:48:47.907172 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901440 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 03:48:47.907609 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901443 2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 03:48:47.907609 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901445 2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 03:48:47.907609 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901448 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 03:48:47.907609 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901451 2575 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 03:48:47.907609 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901454 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 03:48:47.907609 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901457 2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 03:48:47.907609 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901459 2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 03:48:47.907609 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901462 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 03:48:47.907609 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901465 2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 03:48:47.907609 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901468 2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 03:48:47.907609 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901471 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 03:48:47.907609 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901473 2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 03:48:47.907609 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901477 2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 03:48:47.907609 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901479 2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 03:48:47.907609 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901482 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 03:48:47.907609 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.901513 2575 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 03:48:47.908014 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901658 2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 03:48:47.908014 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901663 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 03:48:47.908014 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901666 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 03:48:47.908014 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901669 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 03:48:47.908014 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901672 2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 03:48:47.908014 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901674 2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 03:48:47.908014 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901678 2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 03:48:47.908014 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901682 2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 03:48:47.908014 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901685 2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 03:48:47.908014 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901688 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 03:48:47.908014 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901690 2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 03:48:47.908014 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901693 2575 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 03:48:47.908014 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901695 2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 03:48:47.908014 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901698 2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 03:48:47.908014 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901700 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 03:48:47.908014 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901703 2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 03:48:47.908014 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901705 2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 03:48:47.908014 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901708 2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 03:48:47.908014 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901711 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 03:48:47.908446 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901713 2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 03:48:47.908446 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901716 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 03:48:47.908446 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901719 2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 03:48:47.908446 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901722 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 03:48:47.908446 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901724 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 03:48:47.908446 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901727 2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 03:48:47.908446 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901729 2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 03:48:47.908446 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901731 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 03:48:47.908446 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901734 2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 03:48:47.908446 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901736 2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 03:48:47.908446 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901739 2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 03:48:47.908446 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901742 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 03:48:47.908446 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901744 2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 03:48:47.908446 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901747 2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 03:48:47.908446 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901750 2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 03:48:47.908446 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901771 2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 03:48:47.908446 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901775 2575 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 03:48:47.908446 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901777 2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 03:48:47.908446 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901780 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 03:48:47.908446 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901782 2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 03:48:47.908948 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901785 2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 03:48:47.908948 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901788 2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 03:48:47.908948 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901790 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 03:48:47.908948 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901793 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 03:48:47.908948 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901796 2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 03:48:47.908948 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901798 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 03:48:47.908948 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901801 2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 03:48:47.908948 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901803 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 03:48:47.908948 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901806 2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 03:48:47.908948 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901809 2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 03:48:47.908948 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901811 2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 03:48:47.908948 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901813 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 03:48:47.908948 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901816 2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 03:48:47.908948 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901819 2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 03:48:47.908948 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901821 2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 03:48:47.908948 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901824 2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 03:48:47.908948 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901827 2575 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 03:48:47.908948 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901831 2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 03:48:47.908948 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901835 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 03:48:47.909438 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901838 2575 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 03:48:47.909438 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901841 2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 03:48:47.909438 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901844 2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 03:48:47.909438 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901846 2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 03:48:47.909438 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901849 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 03:48:47.909438 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901851 2575 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 03:48:47.909438 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901854 2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 03:48:47.909438 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901857 2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 03:48:47.909438 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901860 2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 03:48:47.909438 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901862 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 03:48:47.909438 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901865 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 03:48:47.909438 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901868 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 03:48:47.909438 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901870 2575 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 03:48:47.909438 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901873 2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 03:48:47.909438 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901876 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 03:48:47.909438 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901878 2575 feature_gate.go:386] unrecognized feature gate: Example Nov 04 03:48:47.909438 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901881 2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 03:48:47.909438 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901884 2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 03:48:47.909438 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901887 2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 03:48:47.909438 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901889 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 03:48:47.910006 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901892 2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 03:48:47.910006 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901895 2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 03:48:47.910006 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901897 2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 03:48:47.910006 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901899 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 03:48:47.910006 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901902 2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 03:48:47.910006 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901904 2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 03:48:47.910006 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901907 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 03:48:47.910006 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901910 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 03:48:47.910006 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901912 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 03:48:47.910006 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901915 2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 03:48:47.910006 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901918 2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 03:48:47.910006 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901920 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 03:48:47.910006 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901923 2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 03:48:47.910006 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901926 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 03:48:47.910006 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901928 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 03:48:47.910006 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901931 2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 03:48:47.910006 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901934 2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 03:48:47.910006 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901937 2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 03:48:47.910006 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901939 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 03:48:47.910006 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901942 2575 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 03:48:47.910467 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901945 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 03:48:47.910467 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901947 2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 03:48:47.910467 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901950 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 03:48:47.910467 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901952 2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 03:48:47.910467 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901955 2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 03:48:47.910467 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901957 2575 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 03:48:47.910467 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901960 2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 03:48:47.910467 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.901963 2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 03:48:47.910467 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.901987 2575 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 03:48:47.910467 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902018 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 03:48:47.910467 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902022 2575 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 03:48:47.910467 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902026 2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 03:48:47.910467 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902028 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 03:48:47.910467 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902031 2575 feature_gate.go:386] unrecognized feature gate: Example Nov 04 03:48:47.910467 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902034 2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 03:48:47.910467 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902037 2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 03:48:47.910875 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902039 2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 03:48:47.910875 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902042 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 03:48:47.910875 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902045 2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 03:48:47.910875 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902047 2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 03:48:47.910875 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902050 2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 03:48:47.910875 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902053 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 03:48:47.910875 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902056 2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 03:48:47.910875 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902058 2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 03:48:47.910875 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902065 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 03:48:47.910875 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902068 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 03:48:47.910875 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902071 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 03:48:47.910875 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902074 2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 03:48:47.910875 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902076 2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 03:48:47.910875 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902079 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 03:48:47.910875 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902081 2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 03:48:47.910875 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902084 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 03:48:47.910875 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902087 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 03:48:47.910875 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902089 2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 03:48:47.910875 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902092 2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 03:48:47.911552 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902095 2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 03:48:47.911552 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902097 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 03:48:47.911552 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902100 2575 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 03:48:47.911552 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902102 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 03:48:47.911552 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902105 2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 03:48:47.911552 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902108 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 03:48:47.911552 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902111 2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 03:48:47.911552 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902113 2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 03:48:47.911552 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902116 2575 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 03:48:47.911552 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902118 2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 03:48:47.911552 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902121 2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 03:48:47.911552 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902124 2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 03:48:47.911552 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902126 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 03:48:47.911552 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902129 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 03:48:47.911552 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902131 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 03:48:47.911552 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902134 2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 03:48:47.911552 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902137 2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 03:48:47.911552 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902141 2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 03:48:47.911552 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902145 2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 03:48:47.912283 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902148 2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 03:48:47.912283 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902156 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 03:48:47.912283 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902162 2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 03:48:47.912283 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902166 2575 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 03:48:47.912283 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902169 2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 03:48:47.912283 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902172 2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 03:48:47.912283 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902174 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 03:48:47.912283 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902177 2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 03:48:47.912283 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902180 2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 03:48:47.912283 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902182 2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 03:48:47.912283 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902185 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 03:48:47.912283 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902187 2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 03:48:47.912283 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902190 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 03:48:47.912283 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902192 2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 03:48:47.912283 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902195 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 03:48:47.912283 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902198 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 03:48:47.912283 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902200 2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 03:48:47.912283 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902203 2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 03:48:47.912283 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902205 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 03:48:47.912283 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902208 2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 03:48:47.912886 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902210 2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 03:48:47.912886 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902213 2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 03:48:47.912886 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902215 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 03:48:47.912886 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902218 2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 03:48:47.912886 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902220 2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 03:48:47.912886 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902223 2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 03:48:47.912886 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902226 2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 03:48:47.912886 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902228 2575 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 03:48:47.912886 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902232 2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 03:48:47.912886 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902235 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 03:48:47.912886 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902237 2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 03:48:47.912886 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902240 2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 03:48:47.912886 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902242 2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 03:48:47.912886 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902245 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 03:48:47.912886 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902249 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 03:48:47.912886 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902252 2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 03:48:47.912886 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902255 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 03:48:47.912886 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902258 2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 03:48:47.912886 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902260 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 03:48:47.912886 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902263 2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 03:48:47.913400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902265 2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 03:48:47.913400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902268 2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 03:48:47.913400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902270 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 03:48:47.913400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902273 2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 03:48:47.913400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902276 2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 03:48:47.913400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902279 2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 03:48:47.913400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902281 2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 03:48:47.913400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902284 2575 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 03:48:47.913400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902288 2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 03:48:47.913400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902291 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 03:48:47.913400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902296 2575 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 03:48:47.913400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902298 2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 03:48:47.913400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902301 2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 03:48:47.913400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902304 2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 03:48:47.913400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902307 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 03:48:47.913400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902310 2575 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 03:48:47.913400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902313 2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 03:48:47.913400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902315 2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 03:48:47.913400 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902318 2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 03:48:47.913856 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902320 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 03:48:47.913856 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:47.902325 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 03:48:47.913856 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.902389 2575 server.go:957] "Client rotation is on, will bootstrap in background" Nov 04 03:48:47.913856 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.904281 2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Nov 04 03:48:47.913856 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.905184 2575 server.go:1014] "Starting client certificate rotation" Nov 04 03:48:47.913856 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.905286 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 03:48:47.913856 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.905323 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 03:48:47.913856 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.907124 2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 03:48:47.913856 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.912928 2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 03:48:47.922333 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.922310 2575 log.go:25] "Validated CRI v1 runtime API" Nov 04 03:48:47.927296 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.927276 2575 log.go:25] "Validated CRI v1 image API" Nov 04 03:48:47.928176 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.928147 2575 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Nov 04 03:48:47.932550 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.932527 2575 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7b3a2a3f-ff03-47f0-b018-938c8d5fbee9:/dev/nvme0n1p3 944ac331-65b0-4051-ae09-e6420eb62285:/dev/nvme0n1p4] Nov 04 03:48:47.932654 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.932548 2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Nov 04 03:48:47.937880 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.937743 2575 manager.go:217] Machine: {Timestamp:2026-11-04 03:48:47.936377826 +0000 UTC m=+0.305838493 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3126839 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20bee300a0e6f49cecca986af8fa37 SystemUUID:ec20bee3-00a0-e6f4-9cec-ca986af8fa37 BootID:40862e3d-e54b-4230-96be-d63a9aff4b15 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:eb:18:42:4f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:eb:18:42:4f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:33:b6:44:96:0b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Nov 04 03:48:47.937880 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.937880 2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Nov 04 03:48:47.938025 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.938012 2575 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Nov 04 03:48:47.941521 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.941494 2575 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Nov 04 03:48:47.941680 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.941524 2575 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-134.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Nov 04 03:48:47.941725 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.941682 2575 topology_manager.go:143] "Creating topology manager with none policy" Nov 04 03:48:47.941725 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.941691 2575 container_manager_linux.go:311] "Creating device plugin manager" Nov 04 03:48:47.941725 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.941717 2575 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 03:48:47.941829 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.941730 2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 03:48:47.942417 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.942406 2575 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Nov 04 03:48:47.943144 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.943124 2575 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Nov 04 03:48:47.946191 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.946178 2575 state_mem.go:41] "Initialized" logger="CPUManager state memory" Nov 04 03:48:47.946326 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.946317 2575 server.go:1235] "Using root directory" path="/var/lib/kubelet" Nov 04 03:48:47.946362 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.946352 2575 kubelet.go:493] "Attempting to sync node with API server" Nov 04 03:48:47.946403 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.946368 2575 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Nov 04 03:48:47.946403 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.946379 2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Nov 04 03:48:47.946403 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.946389 2575 kubelet.go:405] "Adding apiserver pod source" Nov 04 03:48:47.946403 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.946397 2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Nov 04 03:48:47.946814 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.946801 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 03:48:47.949156 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.949132 2575 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Nov 04 03:48:47.953816 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.953798 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 03:48:47.954840 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.954815 2575 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Nov 04 03:48:47.954930 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.954869 2575 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Nov 04 03:48:47.956523 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.956508 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Nov 04 03:48:47.956586 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.956528 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Nov 04 03:48:47.956586 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.956535 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Nov 04 03:48:47.956586 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.956541 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Nov 04 03:48:47.956586 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.956548 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Nov 04 03:48:47.956586 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.956554 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Nov 04 03:48:47.956586 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.956560 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Nov 04 03:48:47.956586 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.956566 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Nov 04 03:48:47.956586 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.956574 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Nov 04 03:48:47.956586 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.956580 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Nov 04 03:48:47.956586 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.956589 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Nov 04 03:48:47.956904 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.956598 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Nov 04 03:48:47.956904 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.956622 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Nov 04 03:48:47.956904 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.956628 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Nov 04 03:48:47.960493 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.960478 2575 server.go:1263] "Started kubelet" Nov 04 03:48:47.960588 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.960556 2575 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Nov 04 03:48:47.960674 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.960633 2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Nov 04 03:48:47.960714 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.960695 2575 server_v1.go:49] "podresources" method="list" useActivePods=true Nov 04 03:48:47.961736 ip-10-0-129-134 systemd[1]: Started Kubernetes Kubelet. Nov 04 03:48:47.961987 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.961874 2575 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Nov 04 03:48:47.961987 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.961881 2575 server.go:317] "Adding debug handlers to kubelet server" Nov 04 03:48:47.964247 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.964225 2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-134.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 04 03:48:47.967867 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:47.967748 2575 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Nov 04 03:48:47.968259 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.968236 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Nov 04 03:48:47.968732 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.968713 2575 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Nov 04 03:48:47.968982 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:47.968966 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-134.us-east-2.compute.internal\" not found" Nov 04 03:48:47.969224 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.969203 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jfz7s" Nov 04 03:48:47.969320 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:47.969307 2575 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" Nov 04 03:48:47.969437 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.969418 2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Nov 04 03:48:47.969437 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:47.969418 2575 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-134.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" Nov 04 03:48:47.969437 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.969425 2575 volume_manager.go:309] "The desired_state_of_world populator starts" Nov 04 03:48:47.969612 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.969446 2575 volume_manager.go:311] "Starting Kubelet Volume Manager" Nov 04 03:48:47.969612 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.969553 2575 reconstruct.go:97] "Volume reconstruction finished" Nov 04 03:48:47.969612 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.969559 2575 reconciler.go:29] "Reconciler: start to sync state" Nov 04 03:48:47.969727 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.969666 2575 factory.go:55] Registering systemd factory Nov 04 03:48:47.969727 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.969680 2575 factory.go:223] Registration of the systemd container factory successfully Nov 04 03:48:47.969915 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.969903 2575 factory.go:153] Registering CRI-O factory Nov 04 03:48:47.969915 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.969917 2575 factory.go:223] Registration of the crio container factory successfully Nov 04 03:48:47.970020 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.969965 2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Nov 04 03:48:47.970020 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.969980 2575 factory.go:103] Registering Raw factory Nov 04 03:48:47.970020 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.969991 2575 manager.go:1199] Started watching for new ooms in manager Nov 04 03:48:47.970229 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:47.969297 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-134.us-east-2.compute.internal.18e4baf879ebc375 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-134.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-134.us-east-2.compute.internal,},FirstTimestamp:2026-11-04 03:48:47.960449909 +0000 UTC m=+0.329910576,LastTimestamp:2026-11-04 03:48:47.960449909 +0000 UTC m=+0.329910576,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-134.us-east-2.compute.internal,}" Nov 04 03:48:47.971288 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.971256 2575 manager.go:319] Starting recovery of all containers Nov 04 03:48:47.971570 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:47.971547 2575 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-134.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Nov 04 03:48:47.978893 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:47.978862 2575 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" Nov 04 03:48:47.981734 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.981705 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jfz7s" Nov 04 03:48:47.981949 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.981925 2575 manager.go:324] Recovery completed Nov 04 03:48:47.988341 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.988326 2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 03:48:47.990190 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.990175 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-134.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 03:48:47.990265 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.990204 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-134.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 03:48:47.990265 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.990217 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-134.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 04 03:48:47.990850 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.990835 2575 cpu_manager.go:226] "Starting" policy="none" Nov 04 03:48:47.990850 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.990848 2575 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Nov 04 03:48:47.990850 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.990860 2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 03:48:47.990850 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.990869 2575 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Nov 04 03:48:47.992127 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.992106 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-134.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 03:48:47.992184 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.992142 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-134.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 03:48:47.992184 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.992157 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-134.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 04 03:48:47.992517 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:47.992446 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-134.us-east-2.compute.internal.18e4baf87bb18bfb default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-134.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-134.us-east-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-134.us-east-2.compute.internal,},FirstTimestamp:2026-11-04 03:48:47.990189051 +0000 UTC m=+0.359649718,LastTimestamp:2026-11-04 03:48:47.990189051 +0000 UTC m=+0.359649718,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-134.us-east-2.compute.internal,}" Nov 04 03:48:47.992924 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.992913 2575 policy_none.go:50] "Start" Nov 04 03:48:47.992962 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.992929 2575 memory_manager.go:187] "Starting memorymanager" policy="None" Nov 04 03:48:47.992962 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.992939 2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Nov 04 03:48:47.995093 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:47.994974 2575 policy_none.go:44] "Start" Nov 04 03:48:48.040368 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.040331 2575 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Nov 04 03:48:48.043109 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.043093 2575 manager.go:341] "Starting Device Plugin manager" Nov 04 03:48:48.043649 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:48.043625 2575 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Nov 04 03:48:48.043649 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.043647 2575 server.go:87] "Starting device plugin registration server" Nov 04 03:48:48.043912 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.043898 2575 eviction_manager.go:194] "Eviction manager: starting control loop" Nov 04 03:48:48.043990 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.043910 2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Nov 04 03:48:48.044069 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.044051 2575 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Nov 04 03:48:48.044151 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.044134 2575 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Nov 04 03:48:48.044151 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.044147 2575 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Nov 04 03:48:48.044738 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:48.044720 2575 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Nov 04 03:48:48.044843 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:48.044774 2575 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-134.us-east-2.compute.internal\" not found" Nov 04 03:48:48.144097 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.144020 2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 03:48:48.145299 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.145276 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-134.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 03:48:48.145384 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.145315 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-134.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 03:48:48.145384 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.145329 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-134.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 04 03:48:48.145384 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.145357 2575 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:48:48.154202 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.154178 2575 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:48:48.154303 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:48.154212 2575 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-134.us-east-2.compute.internal\": node \"ip-10-0-129-134.us-east-2.compute.internal\" not found" Nov 04 03:48:48.162532 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.162505 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Nov 04 03:48:48.163792 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.163775 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Nov 04 03:48:48.163859 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.163799 2575 status_manager.go:249] "Starting to sync pod status with apiserver" Nov 04 03:48:48.163859 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.163818 2575 kubelet.go:2521] "Starting kubelet main sync loop" Nov 04 03:48:48.163859 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:48.163851 2575 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Nov 04 03:48:48.170140 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.170110 2575 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 03:48:48.178297 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:48.178277 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-134.us-east-2.compute.internal\" not found" Nov 04 03:48:48.264187 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.264140 2575 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal"] Nov 04 03:48:48.264286 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.264231 2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 03:48:48.265331 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.265312 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-134.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 03:48:48.265403 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.265350 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-134.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 03:48:48.265403 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.265364 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-134.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 04 03:48:48.267566 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.267553 2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 03:48:48.267725 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.267707 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:48:48.267772 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.267742 2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 03:48:48.268479 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.268460 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-134.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 03:48:48.268595 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.268494 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-134.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 03:48:48.268595 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.268505 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-134.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 04 03:48:48.268595 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.268466 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-134.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 03:48:48.268595 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.268580 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-134.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 03:48:48.268595 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.268595 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-134.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 04 03:48:48.270798 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.270782 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:48:48.270883 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.270814 2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 03:48:48.271737 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.271722 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-134.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 03:48:48.271844 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.271751 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-134.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 03:48:48.271844 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.271785 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-134.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 04 03:48:48.278932 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:48.278914 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-134.us-east-2.compute.internal\" not found" Nov 04 03:48:48.303913 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:48.303888 2575 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-134.us-east-2.compute.internal\" not found" node="ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:48:48.306706 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:48.306682 2575 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-134.us-east-2.compute.internal\" not found" node="ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:48:48.371377 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.371333 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/295feeff6b2d9f63532642beaf61c650-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal\" (UID: \"295feeff6b2d9f63532642beaf61c650\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:48:48.371377 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.371378 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/18d40cab8244943c9879207cd43e3bc3-config\") pod \"kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal\" (UID: \"18d40cab8244943c9879207cd43e3bc3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:48:48.371590 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.371394 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/295feeff6b2d9f63532642beaf61c650-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal\" (UID: \"295feeff6b2d9f63532642beaf61c650\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:48:48.379468 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:48.379436 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-134.us-east-2.compute.internal\" not found" Nov 04 03:48:48.471872 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.471804 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/295feeff6b2d9f63532642beaf61c650-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal\" (UID: \"295feeff6b2d9f63532642beaf61c650\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:48:48.471872 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.471854 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/295feeff6b2d9f63532642beaf61c650-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal\" (UID: \"295feeff6b2d9f63532642beaf61c650\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:48:48.471872 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.471873 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/18d40cab8244943c9879207cd43e3bc3-config\") pod \"kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal\" (UID: \"18d40cab8244943c9879207cd43e3bc3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:48:48.472000 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.471902 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/295feeff6b2d9f63532642beaf61c650-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal\" (UID: \"295feeff6b2d9f63532642beaf61c650\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:48:48.472000 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.471934 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/18d40cab8244943c9879207cd43e3bc3-config\") pod \"kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal\" (UID: \"18d40cab8244943c9879207cd43e3bc3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:48:48.472000 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.471936 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/295feeff6b2d9f63532642beaf61c650-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal\" (UID: \"295feeff6b2d9f63532642beaf61c650\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:48:48.479895 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:48.479875 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-134.us-east-2.compute.internal\" not found" Nov 04 03:48:48.580444 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:48.580401 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-134.us-east-2.compute.internal\" not found" Nov 04 03:48:48.606604 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.606576 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:48:48.609816 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.609780 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:48:48.681162 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:48.681126 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-134.us-east-2.compute.internal\" not found" Nov 04 03:48:48.782232 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:48.782149 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-134.us-east-2.compute.internal\" not found" Nov 04 03:48:48.865091 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.865058 2575 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 03:48:48.882831 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:48.882796 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-134.us-east-2.compute.internal\" not found" Nov 04 03:48:48.905683 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.905638 2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Nov 04 03:48:48.905848 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.905829 2575 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Nov 04 03:48:48.905896 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.905851 2575 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Nov 04 03:48:48.983662 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:48.983629 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-134.us-east-2.compute.internal\" not found" Nov 04 03:48:48.988281 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.988251 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 03:43:48 +0000 UTC" deadline="2028-05-17 00:30:16.153939073 +0000 UTC" Nov 04 03:48:48.988390 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:48.988283 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13436h41m27.165662575s" Nov 04 03:48:49.016746 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.016716 2575 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 03:48:49.084192 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:49.084152 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-134.us-east-2.compute.internal\" not found" Nov 04 03:48:49.185178 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:49.185142 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-134.us-east-2.compute.internal\" not found" Nov 04 03:48:49.286048 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:49.286007 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-134.us-east-2.compute.internal\" not found" Nov 04 03:48:49.342899 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:49.342815 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18d40cab8244943c9879207cd43e3bc3.slice/crio-1ab58e8cd93b2dc15d1ce30204828c3d59ecad172948a145590ac93dec61b14d WatchSource:0}: Error finding container 1ab58e8cd93b2dc15d1ce30204828c3d59ecad172948a145590ac93dec61b14d: Status 404 returned error can't find the container with id 1ab58e8cd93b2dc15d1ce30204828c3d59ecad172948a145590ac93dec61b14d Nov 04 03:48:49.349495 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.349475 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 03:48:49.353739 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:49.353714 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod295feeff6b2d9f63532642beaf61c650.slice/crio-7ead774dcedf1f778aa4931d15c7a1dd22b1748a24133f95dac09e4a3f393764 WatchSource:0}: Error finding container 7ead774dcedf1f778aa4931d15c7a1dd22b1748a24133f95dac09e4a3f393764: Status 404 returned error can't find the container with id 7ead774dcedf1f778aa4931d15c7a1dd22b1748a24133f95dac09e4a3f393764 Nov 04 03:48:49.387054 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:49.387019 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-134.us-east-2.compute.internal\" not found" Nov 04 03:48:49.487345 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:49.487304 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-134.us-east-2.compute.internal\" not found" Nov 04 03:48:49.573146 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.573121 2575 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 03:48:49.673061 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.672970 2575 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:48:49.686200 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.686171 2575 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 03:48:49.687169 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.687154 2575 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:48:49.701767 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.701724 2575 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 03:48:49.947947 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.947873 2575 apiserver.go:52] "Watching apiserver" Nov 04 03:48:49.959375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.959342 2575 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Nov 04 03:48:49.959738 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.959712 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-5kmqc","kube-system/konnectivity-agent-895vv","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59","openshift-multus/multus-additional-cni-plugins-lrdgr","openshift-network-operator/iptables-alerter-mm4gp","openshift-image-registry/node-ca-7vkwm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal","openshift-network-diagnostics/network-check-target-967tl","openshift-ovn-kubernetes/ovnkube-node-zfgzh","openshift-dns/node-resolver-dxmg7","openshift-multus/network-metrics-daemon-p422k","kube-system/global-pull-secret-syncer-p84rx","kube-system/kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal","openshift-cluster-node-tuning-operator/tuned-zgj8w"] Nov 04 03:48:49.963467 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.962838 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dxmg7" Nov 04 03:48:49.965219 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.965189 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:48:49.965354 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:49.965275 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p422k" podUID="56348938-ef44-41ab-8a71-acc1e4308262" Nov 04 03:48:49.967174 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.967154 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-x8rgd\"" Nov 04 03:48:49.967289 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.967250 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Nov 04 03:48:49.967350 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.967338 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Nov 04 03:48:49.968422 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.967843 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:48:49.968898 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.968881 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Nov 04 03:48:49.970980 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.970959 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:48:49.973555 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.973534 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mm4gp" Nov 04 03:48:49.974080 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.974059 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Nov 04 03:48:49.976009 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.975992 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7vkwm" Nov 04 03:48:49.978684 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.978659 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:49.980239 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.980213 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:48:49.980403 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.980257 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:48:49.980403 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.980289 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zpwv\" (UniqueName: \"kubernetes.io/projected/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-kube-api-access-4zpwv\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:48:49.980403 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.980354 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b70504f-4169-44b2-8b4e-979a068f070d-tmp-dir\") pod \"node-resolver-dxmg7\" (UID: \"2b70504f-4169-44b2-8b4e-979a068f070d\") " pod="openshift-dns/node-resolver-dxmg7" Nov 04 03:48:49.980403 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.980378 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txw7k\" (UniqueName: \"kubernetes.io/projected/2b70504f-4169-44b2-8b4e-979a068f070d-kube-api-access-txw7k\") pod \"node-resolver-dxmg7\" (UID: \"2b70504f-4169-44b2-8b4e-979a068f070d\") " pod="openshift-dns/node-resolver-dxmg7" Nov 04 03:48:49.980624 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.980489 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2b70504f-4169-44b2-8b4e-979a068f070d-hosts-file\") pod \"node-resolver-dxmg7\" (UID: \"2b70504f-4169-44b2-8b4e-979a068f070d\") " pod="openshift-dns/node-resolver-dxmg7" Nov 04 03:48:49.980624 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.980533 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs\") pod \"network-metrics-daemon-p422k\" (UID: \"56348938-ef44-41ab-8a71-acc1e4308262\") " pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:48:49.980624 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.980575 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-system-cni-dir\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:48:49.980624 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.980608 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:48:49.980942 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.980639 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-socket-dir\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:48:49.980942 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.980671 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-registration-dir\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:48:49.980942 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.980774 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-device-dir\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:48:49.980942 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.980829 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-etc-selinux\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:48:49.980942 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.980869 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-cnibin\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:48:49.980942 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.980896 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-cni-binary-copy\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:48:49.980942 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.980926 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:48:49.981289 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.980969 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94ws5\" (UniqueName: \"kubernetes.io/projected/56348938-ef44-41ab-8a71-acc1e4308262-kube-api-access-94ws5\") pod \"network-metrics-daemon-p422k\" (UID: \"56348938-ef44-41ab-8a71-acc1e4308262\") " pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:48:49.981289 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.980997 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-sys-fs\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:48:49.981289 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.981023 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftm9l\" (UniqueName: \"kubernetes.io/projected/eaec1015-2b2b-4f06-b41d-028671919cf0-kube-api-access-ftm9l\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:48:49.981439 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.981411 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:49.984107 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.984082 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-895vv" Nov 04 03:48:49.987158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.987137 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5kmqc" Nov 04 03:48:49.988813 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.988752 2575 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Nov 04 03:48:49.990940 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.990920 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:48:49.991044 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:49.991007 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-967tl" podUID="5cf7e388-1678-4afb-8877-30179162401b" Nov 04 03:48:49.993466 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:49.993436 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:48:49.993565 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:49.993511 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p84rx" podUID="06a274ba-93f4-4dbc-ac39-4039fd254e7d" Nov 04 03:48:50.007873 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.007846 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8b4hm" Nov 04 03:48:50.020456 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.020427 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8b4hm" Nov 04 03:48:50.071525 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.071492 2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Nov 04 03:48:50.075676 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.075646 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-twz4j\"" Nov 04 03:48:50.076224 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.076118 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Nov 04 03:48:50.077331 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.077263 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Nov 04 03:48:50.077627 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.077605 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-m5nwx\"" Nov 04 03:48:50.077707 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.077662 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Nov 04 03:48:50.077847 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.077834 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4ttlw\"" Nov 04 03:48:50.077897 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.077886 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Nov 04 03:48:50.078036 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.078019 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Nov 04 03:48:50.078115 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.078083 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Nov 04 03:48:50.078418 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.078399 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tl6ng\"" Nov 04 03:48:50.078493 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.078477 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Nov 04 03:48:50.078548 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.078531 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Nov 04 03:48:50.078603 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.078571 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-78gsl\"" Nov 04 03:48:50.078656 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.078620 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Nov 04 03:48:50.078710 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.078666 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-qdtd9\"" Nov 04 03:48:50.078710 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.078706 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Nov 04 03:48:50.078835 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.078738 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Nov 04 03:48:50.078835 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.078783 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Nov 04 03:48:50.079271 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.079230 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-j5cz2\"" Nov 04 03:48:50.079340 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.079295 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Nov 04 03:48:50.079340 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.079308 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Nov 04 03:48:50.079422 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.079347 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Nov 04 03:48:50.079924 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.079902 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Nov 04 03:48:50.080212 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.080191 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Nov 04 03:48:50.080299 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.080221 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Nov 04 03:48:50.080299 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.080237 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Nov 04 03:48:50.080299 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.080263 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Nov 04 03:48:50.080470 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.080301 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Nov 04 03:48:50.080470 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.080315 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Nov 04 03:48:50.080470 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.080325 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Nov 04 03:48:50.080470 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.080409 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Nov 04 03:48:50.080470 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.080450 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-75zvg\"" Nov 04 03:48:50.081751 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.081723 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94ws5\" (UniqueName: \"kubernetes.io/projected/56348938-ef44-41ab-8a71-acc1e4308262-kube-api-access-94ws5\") pod \"network-metrics-daemon-p422k\" (UID: \"56348938-ef44-41ab-8a71-acc1e4308262\") " pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:48:50.081751 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.081756 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftm9l\" (UniqueName: \"kubernetes.io/projected/eaec1015-2b2b-4f06-b41d-028671919cf0-kube-api-access-ftm9l\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:48:50.081942 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.081788 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-multus-conf-dir\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.081942 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.081815 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:48:50.081942 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.081861 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-run-netns\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.081942 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.081880 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/42510c3d-0283-4901-9d23-a84793598ac9-env-overrides\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.081942 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.081895 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/06a274ba-93f4-4dbc-ac39-4039fd254e7d-dbus\") pod \"global-pull-secret-syncer-p84rx\" (UID: \"06a274ba-93f4-4dbc-ac39-4039fd254e7d\") " pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:48:50.081942 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.081912 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68d54\" (UniqueName: \"kubernetes.io/projected/5bdb7796-b1a3-49a5-81cd-c8101320e172-kube-api-access-68d54\") pod \"iptables-alerter-mm4gp\" (UID: \"5bdb7796-b1a3-49a5-81cd-c8101320e172\") " pod="openshift-network-operator/iptables-alerter-mm4gp" Nov 04 03:48:50.081942 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.081929 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-multus-cni-dir\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.081942 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.081944 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-etc-kubernetes\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.082260 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.081962 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-kubernetes\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.082260 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.081980 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-sysctl-conf\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.082260 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.081996 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-lib-modules\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.082260 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082012 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-etc-selinux\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:48:50.082260 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082028 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8-host\") pod \"node-ca-7vkwm\" (UID: \"e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8\") " pod="openshift-image-registry/node-ca-7vkwm" Nov 04 03:48:50.082260 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082046 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-log-socket\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.082260 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082068 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/42510c3d-0283-4901-9d23-a84793598ac9-ovnkube-script-lib\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.082260 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082085 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vpwl\" (UniqueName: \"kubernetes.io/projected/42510c3d-0283-4901-9d23-a84793598ac9-kube-api-access-9vpwl\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.082260 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082101 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5bdb7796-b1a3-49a5-81cd-c8101320e172-host-slash\") pod \"iptables-alerter-mm4gp\" (UID: \"5bdb7796-b1a3-49a5-81cd-c8101320e172\") " pod="openshift-network-operator/iptables-alerter-mm4gp" Nov 04 03:48:50.082260 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082123 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-modprobe-d\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.082260 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082144 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-cnibin\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.082260 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082163 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-kubelet\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.082260 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082179 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkcvz\" (UniqueName: \"kubernetes.io/projected/e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8-kube-api-access-gkcvz\") pod \"node-ca-7vkwm\" (UID: \"e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8\") " pod="openshift-image-registry/node-ca-7vkwm" Nov 04 03:48:50.082260 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082195 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5bdb7796-b1a3-49a5-81cd-c8101320e172-iptables-alerter-script\") pod \"iptables-alerter-mm4gp\" (UID: \"5bdb7796-b1a3-49a5-81cd-c8101320e172\") " pod="openshift-network-operator/iptables-alerter-mm4gp" Nov 04 03:48:50.082260 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082210 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-cni-binary-copy\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.082260 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082226 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-multus-socket-dir-parent\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.082260 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082242 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-run-netns\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.083063 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082267 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-run-multus-certs\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.083063 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082293 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zpwv\" (UniqueName: \"kubernetes.io/projected/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-kube-api-access-4zpwv\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:48:50.083063 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082309 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b70504f-4169-44b2-8b4e-979a068f070d-tmp-dir\") pod \"node-resolver-dxmg7\" (UID: \"2b70504f-4169-44b2-8b4e-979a068f070d\") " pod="openshift-dns/node-resolver-dxmg7" Nov 04 03:48:50.083063 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082325 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-run-ovn-kubernetes\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.083063 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082344 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-cni-bin\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.083063 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082361 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-host\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.083063 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082376 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-run-k8s-cni-cncf-io\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.083063 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082414 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-var-lib-cni-multus\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.083063 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082434 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2b70504f-4169-44b2-8b4e-979a068f070d-hosts-file\") pod \"node-resolver-dxmg7\" (UID: \"2b70504f-4169-44b2-8b4e-979a068f070d\") " pod="openshift-dns/node-resolver-dxmg7" Nov 04 03:48:50.083063 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082452 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs\") pod \"network-metrics-daemon-p422k\" (UID: \"56348938-ef44-41ab-8a71-acc1e4308262\") " pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:48:50.083063 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082476 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-var-lib-openvswitch\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.083063 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082493 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-run-openvswitch\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.083063 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082516 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.083063 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082532 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-sysctl-d\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.083063 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082579 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-system-cni-dir\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.083063 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082606 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lvzv\" (UniqueName: \"kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv\") pod \"network-check-target-967tl\" (UID: \"5cf7e388-1678-4afb-8877-30179162401b\") " pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:48:50.083884 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082626 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-systemd-units\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.083884 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082652 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-system-cni-dir\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:48:50.083884 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082670 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-registration-dir\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:48:50.083884 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082688 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8-serviceca\") pod \"node-ca-7vkwm\" (UID: \"e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8\") " pod="openshift-image-registry/node-ca-7vkwm" Nov 04 03:48:50.083884 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082712 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-run-ovn\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.083884 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082737 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e07cc7a-88c6-4e29-b7ab-c078916cd977-tmp\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.083884 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082753 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b9rr\" (UniqueName: \"kubernetes.io/projected/3e07cc7a-88c6-4e29-b7ab-c078916cd977-kube-api-access-4b9rr\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.083884 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082780 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dde9ca65-5089-450e-97cd-89bf66adba30-konnectivity-ca\") pod \"konnectivity-agent-895vv\" (UID: \"dde9ca65-5089-450e-97cd-89bf66adba30\") " pod="kube-system/konnectivity-agent-895vv" Nov 04 03:48:50.083884 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082800 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret\") pod \"global-pull-secret-syncer-p84rx\" (UID: \"06a274ba-93f4-4dbc-ac39-4039fd254e7d\") " pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:48:50.083884 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082818 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-cni-binary-copy\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:48:50.083884 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082850 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:48:50.083884 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082872 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-cni-netd\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.083884 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082890 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/42510c3d-0283-4901-9d23-a84793598ac9-ovnkube-config\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.083884 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082913 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-tuned\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.083884 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082938 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txw7k\" (UniqueName: \"kubernetes.io/projected/2b70504f-4169-44b2-8b4e-979a068f070d-kube-api-access-txw7k\") pod \"node-resolver-dxmg7\" (UID: \"2b70504f-4169-44b2-8b4e-979a068f070d\") " pod="openshift-dns/node-resolver-dxmg7" Nov 04 03:48:50.083884 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082962 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/06a274ba-93f4-4dbc-ac39-4039fd254e7d-kubelet-config\") pod \"global-pull-secret-syncer-p84rx\" (UID: \"06a274ba-93f4-4dbc-ac39-4039fd254e7d\") " pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:48:50.084648 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082978 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-var-lib-kubelet\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.084648 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.082994 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-multus-daemon-config\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.084648 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.083010 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-sysconfig\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.084648 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.083026 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-systemd\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.084648 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.083048 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-sys\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.084648 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.083076 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-socket-dir\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:48:50.084648 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.083102 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-device-dir\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:48:50.084648 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.083124 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-run\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.084648 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.083142 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-var-lib-kubelet\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.084648 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.083163 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-sys-fs\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:48:50.084648 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.083180 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-etc-openvswitch\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.084648 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.083207 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:48:50.084648 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.083230 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-run-systemd\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.084648 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.083246 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dde9ca65-5089-450e-97cd-89bf66adba30-agent-certs\") pod \"konnectivity-agent-895vv\" (UID: \"dde9ca65-5089-450e-97cd-89bf66adba30\") " pod="kube-system/konnectivity-agent-895vv" Nov 04 03:48:50.084648 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.083264 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-node-log\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.084648 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.083282 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/42510c3d-0283-4901-9d23-a84793598ac9-ovn-node-metrics-cert\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.084648 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.083307 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:48:50.085586 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.083324 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-var-lib-cni-bin\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.085586 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.083340 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-hostroot\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.085586 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.083356 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n49rr\" (UniqueName: \"kubernetes.io/projected/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-kube-api-access-n49rr\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.085586 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.083379 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-slash\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.085586 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.083401 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-cnibin\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:48:50.085586 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.083462 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-cnibin\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:48:50.085586 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.084430 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:48:50.085586 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.084654 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-etc-selinux\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:48:50.085586 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.085164 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b70504f-4169-44b2-8b4e-979a068f070d-tmp-dir\") pod \"node-resolver-dxmg7\" (UID: \"2b70504f-4169-44b2-8b4e-979a068f070d\") " pod="openshift-dns/node-resolver-dxmg7" Nov 04 03:48:50.085586 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.085279 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2b70504f-4169-44b2-8b4e-979a068f070d-hosts-file\") pod \"node-resolver-dxmg7\" (UID: \"2b70504f-4169-44b2-8b4e-979a068f070d\") " pod="openshift-dns/node-resolver-dxmg7" Nov 04 03:48:50.085586 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:50.085338 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 03:48:50.085586 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:50.085385 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs podName:56348938-ef44-41ab-8a71-acc1e4308262 nodeName:}" failed. No retries permitted until 2026-11-04 03:48:50.585371003 +0000 UTC m=+2.954831677 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs") pod "network-metrics-daemon-p422k" (UID: "56348938-ef44-41ab-8a71-acc1e4308262") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 03:48:50.086207 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.085622 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-system-cni-dir\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:48:50.086207 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.085672 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-registration-dir\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:48:50.086207 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.086113 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-cni-binary-copy\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:48:50.086359 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.086232 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:48:50.086603 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.086582 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-socket-dir\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:48:50.086679 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.086654 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-device-dir\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:48:50.086825 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.086735 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-sys-fs\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:48:50.087285 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.087148 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:48:50.087749 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.087723 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:48:50.107844 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.107800 2575 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Nov 04 03:48:50.112186 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.112156 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftm9l\" (UniqueName: \"kubernetes.io/projected/eaec1015-2b2b-4f06-b41d-028671919cf0-kube-api-access-ftm9l\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:48:50.116103 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.116065 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94ws5\" (UniqueName: \"kubernetes.io/projected/56348938-ef44-41ab-8a71-acc1e4308262-kube-api-access-94ws5\") pod \"network-metrics-daemon-p422k\" (UID: \"56348938-ef44-41ab-8a71-acc1e4308262\") " pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:48:50.117505 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.117481 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txw7k\" (UniqueName: \"kubernetes.io/projected/2b70504f-4169-44b2-8b4e-979a068f070d-kube-api-access-txw7k\") pod \"node-resolver-dxmg7\" (UID: \"2b70504f-4169-44b2-8b4e-979a068f070d\") " pod="openshift-dns/node-resolver-dxmg7" Nov 04 03:48:50.118182 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.118158 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zpwv\" (UniqueName: \"kubernetes.io/projected/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-kube-api-access-4zpwv\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:48:50.174471 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.174388 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" event={"ID":"295feeff6b2d9f63532642beaf61c650","Type":"ContainerStarted","Data":"7ead774dcedf1f778aa4931d15c7a1dd22b1748a24133f95dac09e4a3f393764"} Nov 04 03:48:50.175898 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.175864 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal" event={"ID":"18d40cab8244943c9879207cd43e3bc3","Type":"ContainerStarted","Data":"1ab58e8cd93b2dc15d1ce30204828c3d59ecad172948a145590ac93dec61b14d"} Nov 04 03:48:50.184355 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.184304 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5bdb7796-b1a3-49a5-81cd-c8101320e172-host-slash\") pod \"iptables-alerter-mm4gp\" (UID: \"5bdb7796-b1a3-49a5-81cd-c8101320e172\") " pod="openshift-network-operator/iptables-alerter-mm4gp" Nov 04 03:48:50.184490 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.184358 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-modprobe-d\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.184490 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.184440 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5bdb7796-b1a3-49a5-81cd-c8101320e172-host-slash\") pod \"iptables-alerter-mm4gp\" (UID: \"5bdb7796-b1a3-49a5-81cd-c8101320e172\") " pod="openshift-network-operator/iptables-alerter-mm4gp" Nov 04 03:48:50.184593 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.184520 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-cnibin\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.184593 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.184553 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-kubelet\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.184702 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.184596 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkcvz\" (UniqueName: \"kubernetes.io/projected/e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8-kube-api-access-gkcvz\") pod \"node-ca-7vkwm\" (UID: \"e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8\") " pod="openshift-image-registry/node-ca-7vkwm" Nov 04 03:48:50.184702 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.184603 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-modprobe-d\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.184702 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.184677 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5bdb7796-b1a3-49a5-81cd-c8101320e172-iptables-alerter-script\") pod \"iptables-alerter-mm4gp\" (UID: \"5bdb7796-b1a3-49a5-81cd-c8101320e172\") " pod="openshift-network-operator/iptables-alerter-mm4gp" Nov 04 03:48:50.184881 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.184710 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-cni-binary-copy\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.184881 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.184716 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-cnibin\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.184881 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.184736 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-multus-socket-dir-parent\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.184881 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.184762 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-run-netns\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.184881 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.184838 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-run-multus-certs\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.184881 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.184876 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-run-ovn-kubernetes\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.185181 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.184901 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-cni-bin\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.185181 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.184927 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-host\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.185181 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185005 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-run-k8s-cni-cncf-io\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.185181 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185030 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-var-lib-cni-multus\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.185181 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185070 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-var-lib-openvswitch\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.185181 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185109 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-run-openvswitch\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.185181 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185152 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.185541 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185199 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-sysctl-d\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.185541 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185224 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-system-cni-dir\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.185541 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185258 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lvzv\" (UniqueName: \"kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv\") pod \"network-check-target-967tl\" (UID: \"5cf7e388-1678-4afb-8877-30179162401b\") " pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:48:50.185541 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185331 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-systemd-units\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.185541 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185359 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8-serviceca\") pod \"node-ca-7vkwm\" (UID: \"e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8\") " pod="openshift-image-registry/node-ca-7vkwm" Nov 04 03:48:50.185541 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185389 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-run-ovn\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.185541 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185412 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e07cc7a-88c6-4e29-b7ab-c078916cd977-tmp\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.185541 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185458 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b9rr\" (UniqueName: \"kubernetes.io/projected/3e07cc7a-88c6-4e29-b7ab-c078916cd977-kube-api-access-4b9rr\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.185541 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185464 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5bdb7796-b1a3-49a5-81cd-c8101320e172-iptables-alerter-script\") pod \"iptables-alerter-mm4gp\" (UID: \"5bdb7796-b1a3-49a5-81cd-c8101320e172\") " pod="openshift-network-operator/iptables-alerter-mm4gp" Nov 04 03:48:50.185541 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185485 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dde9ca65-5089-450e-97cd-89bf66adba30-konnectivity-ca\") pod \"konnectivity-agent-895vv\" (UID: \"dde9ca65-5089-450e-97cd-89bf66adba30\") " pod="kube-system/konnectivity-agent-895vv" Nov 04 03:48:50.185541 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185537 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret\") pod \"global-pull-secret-syncer-p84rx\" (UID: \"06a274ba-93f4-4dbc-ac39-4039fd254e7d\") " pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:48:50.186111 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185572 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-cni-netd\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.186111 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185600 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/42510c3d-0283-4901-9d23-a84793598ac9-ovnkube-config\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.186111 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185625 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-tuned\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.186111 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185654 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/06a274ba-93f4-4dbc-ac39-4039fd254e7d-kubelet-config\") pod \"global-pull-secret-syncer-p84rx\" (UID: \"06a274ba-93f4-4dbc-ac39-4039fd254e7d\") " pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:48:50.186111 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185748 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-var-lib-kubelet\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.186111 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185782 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-multus-daemon-config\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.186111 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185809 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-sysconfig\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.186111 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185858 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-systemd\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.186111 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185884 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-sys\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.186111 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185912 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-run\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.186111 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.185967 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-var-lib-kubelet\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.186111 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.186015 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-etc-openvswitch\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.186111 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.186095 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-run-systemd\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.186665 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.186110 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dde9ca65-5089-450e-97cd-89bf66adba30-konnectivity-ca\") pod \"konnectivity-agent-895vv\" (UID: \"dde9ca65-5089-450e-97cd-89bf66adba30\") " pod="kube-system/konnectivity-agent-895vv" Nov 04 03:48:50.186665 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.186120 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dde9ca65-5089-450e-97cd-89bf66adba30-agent-certs\") pod \"konnectivity-agent-895vv\" (UID: \"dde9ca65-5089-450e-97cd-89bf66adba30\") " pod="kube-system/konnectivity-agent-895vv" Nov 04 03:48:50.186665 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.186176 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-node-log\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.186665 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.186232 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/42510c3d-0283-4901-9d23-a84793598ac9-ovn-node-metrics-cert\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.186665 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.186263 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-var-lib-cni-bin\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.186665 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.186292 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-hostroot\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.186665 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.186319 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n49rr\" (UniqueName: \"kubernetes.io/projected/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-kube-api-access-n49rr\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.186665 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.186347 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-slash\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.186665 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.186392 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-multus-conf-dir\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.186665 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.186481 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-run-netns\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.186665 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.186522 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/42510c3d-0283-4901-9d23-a84793598ac9-env-overrides\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.186665 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.186558 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/06a274ba-93f4-4dbc-ac39-4039fd254e7d-dbus\") pod \"global-pull-secret-syncer-p84rx\" (UID: \"06a274ba-93f4-4dbc-ac39-4039fd254e7d\") " pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:48:50.186665 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.186586 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68d54\" (UniqueName: \"kubernetes.io/projected/5bdb7796-b1a3-49a5-81cd-c8101320e172-kube-api-access-68d54\") pod \"iptables-alerter-mm4gp\" (UID: \"5bdb7796-b1a3-49a5-81cd-c8101320e172\") " pod="openshift-network-operator/iptables-alerter-mm4gp" Nov 04 03:48:50.186665 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.186584 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-sysctl-d\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.187155 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.186675 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-system-cni-dir\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.187155 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.187037 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-systemd-units\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.187155 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.187147 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-multus-cni-dir\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.187300 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.187179 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-etc-kubernetes\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.187300 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.187187 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-hostroot\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.187300 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.187213 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-kubernetes\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.187300 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.187232 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-run-netns\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.187300 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.187284 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-kubelet\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.187553 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.187304 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-run-netns\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.187553 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.187332 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-var-lib-kubelet\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.187553 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.187476 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-sysctl-conf\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.187553 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.187550 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-run-multus-certs\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.187853 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.187604 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-run-ovn-kubernetes\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.187853 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.187647 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-cni-bin\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.187853 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.187696 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-run-k8s-cni-cncf-io\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.187853 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.187742 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-var-lib-cni-multus\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.187853 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.187780 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-var-lib-openvswitch\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.187853 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:50.187808 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 03:48:50.187853 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.187844 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-run-openvswitch\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.187853 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.187848 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-cni-netd\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.188278 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.187906 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.188278 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:50.188041 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret podName:06a274ba-93f4-4dbc-ac39-4039fd254e7d nodeName:}" failed. No retries permitted until 2026-11-04 03:48:50.688002054 +0000 UTC m=+3.057462802 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret") pod "global-pull-secret-syncer-p84rx" (UID: "06a274ba-93f4-4dbc-ac39-4039fd254e7d") : object "kube-system"/"original-pull-secret" not registered Nov 04 03:48:50.188278 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.188110 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-var-lib-cni-bin\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.188278 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.188187 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-sys\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.188278 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.188210 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-sysconfig\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.188278 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.188243 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-systemd\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.188631 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.188291 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-kubernetes\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.188631 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.188338 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-etc-kubernetes\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.188631 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.188377 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/06a274ba-93f4-4dbc-ac39-4039fd254e7d-kubelet-config\") pod \"global-pull-secret-syncer-p84rx\" (UID: \"06a274ba-93f4-4dbc-ac39-4039fd254e7d\") " pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:48:50.188631 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.188613 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-multus-cni-dir\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.188861 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.188730 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/06a274ba-93f4-4dbc-ac39-4039fd254e7d-dbus\") pod \"global-pull-secret-syncer-p84rx\" (UID: \"06a274ba-93f4-4dbc-ac39-4039fd254e7d\") " pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:48:50.189246 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.189187 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-host\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.189409 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.189392 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-var-lib-kubelet\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.189488 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.189463 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-etc-openvswitch\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.189488 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.189424 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-run\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.189614 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.189514 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-multus-socket-dir-parent\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.189614 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.189558 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-run-systemd\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.189722 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.189596 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-slash\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.189722 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.189595 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-multus-conf-dir\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.189722 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.189639 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-node-log\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.189722 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.189637 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-run-ovn\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.189722 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.187243 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-sysctl-conf\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.190656 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.190084 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-cni-binary-copy\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.192105 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.192079 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8-serviceca\") pod \"node-ca-7vkwm\" (UID: \"e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8\") " pod="openshift-image-registry/node-ca-7vkwm" Nov 04 03:48:50.192200 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.192156 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-lib-modules\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.192277 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.192219 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8-host\") pod \"node-ca-7vkwm\" (UID: \"e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8\") " pod="openshift-image-registry/node-ca-7vkwm" Nov 04 03:48:50.192277 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.192267 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-log-socket\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.192386 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.192304 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/42510c3d-0283-4901-9d23-a84793598ac9-ovnkube-script-lib\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.192386 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.192339 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vpwl\" (UniqueName: \"kubernetes.io/projected/42510c3d-0283-4901-9d23-a84793598ac9-kube-api-access-9vpwl\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.192494 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.192441 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/42510c3d-0283-4901-9d23-a84793598ac9-env-overrides\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.192639 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.192614 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-multus-daemon-config\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.192716 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.192664 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/42510c3d-0283-4901-9d23-a84793598ac9-ovnkube-config\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.192766 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.192716 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8-host\") pod \"node-ca-7vkwm\" (UID: \"e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8\") " pod="openshift-image-registry/node-ca-7vkwm" Nov 04 03:48:50.192766 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.192730 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-log-socket\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.192912 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.192879 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-lib-modules\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.193424 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.193213 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dde9ca65-5089-450e-97cd-89bf66adba30-agent-certs\") pod \"konnectivity-agent-895vv\" (UID: \"dde9ca65-5089-450e-97cd-89bf66adba30\") " pod="kube-system/konnectivity-agent-895vv" Nov 04 03:48:50.193424 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.193259 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/42510c3d-0283-4901-9d23-a84793598ac9-ovnkube-script-lib\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.199230 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.199139 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/42510c3d-0283-4901-9d23-a84793598ac9-ovn-node-metrics-cert\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.200490 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.200461 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e07cc7a-88c6-4e29-b7ab-c078916cd977-tmp\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.205784 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.205752 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-tuned\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.232555 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.232514 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkcvz\" (UniqueName: \"kubernetes.io/projected/e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8-kube-api-access-gkcvz\") pod \"node-ca-7vkwm\" (UID: \"e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8\") " pod="openshift-image-registry/node-ca-7vkwm" Nov 04 03:48:50.234687 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.234602 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vpwl\" (UniqueName: \"kubernetes.io/projected/42510c3d-0283-4901-9d23-a84793598ac9-kube-api-access-9vpwl\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.237153 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:50.237128 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 03:48:50.237153 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:50.237156 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 03:48:50.237330 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:50.237172 2575 projected.go:196] Error preparing data for projected volume kube-api-access-8lvzv for pod openshift-network-diagnostics/network-check-target-967tl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 03:48:50.237330 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:50.237254 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv podName:5cf7e388-1678-4afb-8877-30179162401b nodeName:}" failed. No retries permitted until 2026-11-04 03:48:50.737229997 +0000 UTC m=+3.106690684 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8lvzv" (UniqueName: "kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv") pod "network-check-target-967tl" (UID: "5cf7e388-1678-4afb-8877-30179162401b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 03:48:50.238301 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.238273 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68d54\" (UniqueName: \"kubernetes.io/projected/5bdb7796-b1a3-49a5-81cd-c8101320e172-kube-api-access-68d54\") pod \"iptables-alerter-mm4gp\" (UID: \"5bdb7796-b1a3-49a5-81cd-c8101320e172\") " pod="openshift-network-operator/iptables-alerter-mm4gp" Nov 04 03:48:50.238396 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.238310 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n49rr\" (UniqueName: \"kubernetes.io/projected/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-kube-api-access-n49rr\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.239024 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.239003 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b9rr\" (UniqueName: \"kubernetes.io/projected/3e07cc7a-88c6-4e29-b7ab-c078916cd977-kube-api-access-4b9rr\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.277862 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.277827 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dxmg7" Nov 04 03:48:50.288678 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:50.288646 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b70504f_4169_44b2_8b4e_979a068f070d.slice/crio-485212cf3b7bfb927460275c406b97f098b60a53694c113965a2a3cdac372436 WatchSource:0}: Error finding container 485212cf3b7bfb927460275c406b97f098b60a53694c113965a2a3cdac372436: Status 404 returned error can't find the container with id 485212cf3b7bfb927460275c406b97f098b60a53694c113965a2a3cdac372436 Nov 04 03:48:50.291054 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.291031 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:48:50.301045 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:50.301015 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaec1015_2b2b_4f06_b41d_028671919cf0.slice/crio-252bd07aaa5fd2b939fb6a12717ac49ec8facbc3de2f1218886ad60e56a86545 WatchSource:0}: Error finding container 252bd07aaa5fd2b939fb6a12717ac49ec8facbc3de2f1218886ad60e56a86545: Status 404 returned error can't find the container with id 252bd07aaa5fd2b939fb6a12717ac49ec8facbc3de2f1218886ad60e56a86545 Nov 04 03:48:50.310372 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.310335 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:48:50.311120 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.311096 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mm4gp" Nov 04 03:48:50.313702 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.313676 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7vkwm" Nov 04 03:48:50.321607 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:50.321569 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48e3e3bd_2ec1_4587_97b3_7b37fc5130fb.slice/crio-b7658efe8fc795d2864a1422877fa92a46b3c3777d4de011a007d01cce63afc3 WatchSource:0}: Error finding container b7658efe8fc795d2864a1422877fa92a46b3c3777d4de011a007d01cce63afc3: Status 404 returned error can't find the container with id b7658efe8fc795d2864a1422877fa92a46b3c3777d4de011a007d01cce63afc3 Nov 04 03:48:50.325310 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:50.325283 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bdb7796_b1a3_49a5_81cd_c8101320e172.slice/crio-4f15a525e7d518e05acbb46f1a2f234272cc3fbd6461fa94a997007447d95dfa WatchSource:0}: Error finding container 4f15a525e7d518e05acbb46f1a2f234272cc3fbd6461fa94a997007447d95dfa: Status 404 returned error can't find the container with id 4f15a525e7d518e05acbb46f1a2f234272cc3fbd6461fa94a997007447d95dfa Nov 04 03:48:50.331367 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.331336 2575 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 03:48:50.337527 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.337504 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:48:50.343017 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.342997 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:48:50.345724 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.345702 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-895vv" Nov 04 03:48:50.348392 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.348374 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5kmqc" Nov 04 03:48:50.352151 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:50.352118 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e07cc7a_88c6_4e29_b7ab_c078916cd977.slice/crio-791cf65b1f8c8386d936c41485b52bfad3b79489ab5c8d8d680e04041692ce73 WatchSource:0}: Error finding container 791cf65b1f8c8386d936c41485b52bfad3b79489ab5c8d8d680e04041692ce73: Status 404 returned error can't find the container with id 791cf65b1f8c8386d936c41485b52bfad3b79489ab5c8d8d680e04041692ce73 Nov 04 03:48:50.356757 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:50.356732 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42510c3d_0283_4901_9d23_a84793598ac9.slice/crio-3a388a934358f893ce03e72f28185b96d6f68ff1d0423739d397198c220537a7 WatchSource:0}: Error finding container 3a388a934358f893ce03e72f28185b96d6f68ff1d0423739d397198c220537a7: Status 404 returned error can't find the container with id 3a388a934358f893ce03e72f28185b96d6f68ff1d0423739d397198c220537a7 Nov 04 03:48:50.367561 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:48:50.367526 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a0a7a68_52cb_4beb_bb3e_1f5c00e4aa07.slice/crio-c91db1941719fe8a6ca13bb1d85d6b0fd162b50b3f18cfa0e6aa49aa1d331aa6 WatchSource:0}: Error finding container c91db1941719fe8a6ca13bb1d85d6b0fd162b50b3f18cfa0e6aa49aa1d331aa6: Status 404 returned error can't find the container with id c91db1941719fe8a6ca13bb1d85d6b0fd162b50b3f18cfa0e6aa49aa1d331aa6 Nov 04 03:48:50.594906 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.594851 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs\") pod \"network-metrics-daemon-p422k\" (UID: \"56348938-ef44-41ab-8a71-acc1e4308262\") " pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:48:50.595112 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:50.595014 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 03:48:50.595112 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:50.595096 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs podName:56348938-ef44-41ab-8a71-acc1e4308262 nodeName:}" failed. No retries permitted until 2026-11-04 03:48:51.595079169 +0000 UTC m=+3.964539824 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs") pod "network-metrics-daemon-p422k" (UID: "56348938-ef44-41ab-8a71-acc1e4308262") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 03:48:50.695845 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.695802 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret\") pod \"global-pull-secret-syncer-p84rx\" (UID: \"06a274ba-93f4-4dbc-ac39-4039fd254e7d\") " pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:48:50.696041 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:50.695950 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 03:48:50.696041 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:50.696033 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret podName:06a274ba-93f4-4dbc-ac39-4039fd254e7d nodeName:}" failed. No retries permitted until 2026-11-04 03:48:51.696016259 +0000 UTC m=+4.065476915 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret") pod "global-pull-secret-syncer-p84rx" (UID: "06a274ba-93f4-4dbc-ac39-4039fd254e7d") : object "kube-system"/"original-pull-secret" not registered Nov 04 03:48:50.778330 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.778299 2575 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 03:48:50.796750 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:50.796714 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lvzv\" (UniqueName: \"kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv\") pod \"network-check-target-967tl\" (UID: \"5cf7e388-1678-4afb-8877-30179162401b\") " pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:48:50.796933 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:50.796884 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 03:48:50.796933 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:50.796902 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 03:48:50.796933 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:50.796913 2575 projected.go:196] Error preparing data for projected volume kube-api-access-8lvzv for pod openshift-network-diagnostics/network-check-target-967tl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 03:48:50.797035 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:50.796979 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv podName:5cf7e388-1678-4afb-8877-30179162401b nodeName:}" failed. No retries permitted until 2026-11-04 03:48:51.796961827 +0000 UTC m=+4.166422497 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8lvzv" (UniqueName: "kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv") pod "network-check-target-967tl" (UID: "5cf7e388-1678-4afb-8877-30179162401b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 03:48:51.024353 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:51.024180 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 03:43:50 +0000 UTC" deadline="2028-07-31 06:50:58.222762736 +0000 UTC" Nov 04 03:48:51.024353 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:51.024252 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15243h2m7.198515548s" Nov 04 03:48:51.164816 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:51.164757 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:48:51.165007 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:51.164955 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p422k" podUID="56348938-ef44-41ab-8a71-acc1e4308262" Nov 04 03:48:51.183747 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:51.183630 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5kmqc" event={"ID":"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07","Type":"ContainerStarted","Data":"c91db1941719fe8a6ca13bb1d85d6b0fd162b50b3f18cfa0e6aa49aa1d331aa6"} Nov 04 03:48:51.188404 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:51.188366 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" event={"ID":"3e07cc7a-88c6-4e29-b7ab-c078916cd977","Type":"ContainerStarted","Data":"791cf65b1f8c8386d936c41485b52bfad3b79489ab5c8d8d680e04041692ce73"} Nov 04 03:48:51.203919 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:51.203788 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7vkwm" event={"ID":"e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8","Type":"ContainerStarted","Data":"30acd61540bcb6cb1bdbd8114cac25ad46b938f4c4b6516378cf6ab8c8b3623e"} Nov 04 03:48:51.214477 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:51.214374 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lrdgr" event={"ID":"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb","Type":"ContainerStarted","Data":"b7658efe8fc795d2864a1422877fa92a46b3c3777d4de011a007d01cce63afc3"} Nov 04 03:48:51.221960 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:51.221923 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dxmg7" event={"ID":"2b70504f-4169-44b2-8b4e-979a068f070d","Type":"ContainerStarted","Data":"485212cf3b7bfb927460275c406b97f098b60a53694c113965a2a3cdac372436"} Nov 04 03:48:51.226471 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:51.226434 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-895vv" event={"ID":"dde9ca65-5089-450e-97cd-89bf66adba30","Type":"ContainerStarted","Data":"82178b5fdd0242d2074a69d7b974fbc049b8a4bef0605f94ce7089ec8b5347f6"} Nov 04 03:48:51.237570 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:51.237533 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" event={"ID":"42510c3d-0283-4901-9d23-a84793598ac9","Type":"ContainerStarted","Data":"3a388a934358f893ce03e72f28185b96d6f68ff1d0423739d397198c220537a7"} Nov 04 03:48:51.244234 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:51.244198 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mm4gp" event={"ID":"5bdb7796-b1a3-49a5-81cd-c8101320e172","Type":"ContainerStarted","Data":"4f15a525e7d518e05acbb46f1a2f234272cc3fbd6461fa94a997007447d95dfa"} Nov 04 03:48:51.256381 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:51.256343 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" event={"ID":"eaec1015-2b2b-4f06-b41d-028671919cf0","Type":"ContainerStarted","Data":"252bd07aaa5fd2b939fb6a12717ac49ec8facbc3de2f1218886ad60e56a86545"} Nov 04 03:48:51.604179 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:51.603569 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs\") pod \"network-metrics-daemon-p422k\" (UID: \"56348938-ef44-41ab-8a71-acc1e4308262\") " pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:48:51.604179 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:51.603723 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 03:48:51.604179 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:51.603793 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs podName:56348938-ef44-41ab-8a71-acc1e4308262 nodeName:}" failed. No retries permitted until 2026-11-04 03:48:53.603773278 +0000 UTC m=+5.973233939 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs") pod "network-metrics-daemon-p422k" (UID: "56348938-ef44-41ab-8a71-acc1e4308262") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 03:48:51.704296 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:51.704235 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret\") pod \"global-pull-secret-syncer-p84rx\" (UID: \"06a274ba-93f4-4dbc-ac39-4039fd254e7d\") " pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:48:51.704452 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:51.704416 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 03:48:51.704517 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:51.704486 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret podName:06a274ba-93f4-4dbc-ac39-4039fd254e7d nodeName:}" failed. No retries permitted until 2026-11-04 03:48:53.70446588 +0000 UTC m=+6.073926537 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret") pod "global-pull-secret-syncer-p84rx" (UID: "06a274ba-93f4-4dbc-ac39-4039fd254e7d") : object "kube-system"/"original-pull-secret" not registered Nov 04 03:48:51.806099 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:51.805890 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lvzv\" (UniqueName: \"kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv\") pod \"network-check-target-967tl\" (UID: \"5cf7e388-1678-4afb-8877-30179162401b\") " pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:48:51.806275 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:51.806255 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 03:48:51.806363 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:51.806284 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 03:48:51.806363 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:51.806299 2575 projected.go:196] Error preparing data for projected volume kube-api-access-8lvzv for pod openshift-network-diagnostics/network-check-target-967tl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 03:48:51.806363 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:51.806357 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv podName:5cf7e388-1678-4afb-8877-30179162401b nodeName:}" failed. No retries permitted until 2026-11-04 03:48:53.806339329 +0000 UTC m=+6.175800001 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8lvzv" (UniqueName: "kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv") pod "network-check-target-967tl" (UID: "5cf7e388-1678-4afb-8877-30179162401b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 03:48:52.025559 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:52.025392 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 03:43:50 +0000 UTC" deadline="2028-05-04 00:49:49.986480337 +0000 UTC" Nov 04 03:48:52.025559 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:52.025465 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13125h0m57.961019006s" Nov 04 03:48:52.166984 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:52.166952 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:48:52.167200 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:52.167084 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p84rx" podUID="06a274ba-93f4-4dbc-ac39-4039fd254e7d" Nov 04 03:48:52.167577 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:52.167549 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:48:52.167688 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:52.167653 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-967tl" podUID="5cf7e388-1678-4afb-8877-30179162401b" Nov 04 03:48:52.271395 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:52.271355 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal" event={"ID":"18d40cab8244943c9879207cd43e3bc3","Type":"ContainerStarted","Data":"b7436ce2746954cf8f807da767a632dbde5647cd9e00233f79800b9efa5ccff3"} Nov 04 03:48:52.292543 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:52.291439 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal" podStartSLOduration=2.291418766 podStartE2EDuration="2.291418766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:48:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:48:52.290015087 +0000 UTC m=+4.659475772" watchObservedRunningTime="2026-11-04 03:48:52.291418766 +0000 UTC m=+4.660879447" Nov 04 03:48:53.165015 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:53.164890 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:48:53.165504 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:53.165082 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p422k" podUID="56348938-ef44-41ab-8a71-acc1e4308262" Nov 04 03:48:53.623169 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:53.622490 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs\") pod \"network-metrics-daemon-p422k\" (UID: \"56348938-ef44-41ab-8a71-acc1e4308262\") " pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:48:53.623169 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:53.622713 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 03:48:53.623169 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:53.622782 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs podName:56348938-ef44-41ab-8a71-acc1e4308262 nodeName:}" failed. No retries permitted until 2026-11-04 03:48:57.622763324 +0000 UTC m=+9.992223998 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs") pod "network-metrics-daemon-p422k" (UID: "56348938-ef44-41ab-8a71-acc1e4308262") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 03:48:53.723093 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:53.723057 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret\") pod \"global-pull-secret-syncer-p84rx\" (UID: \"06a274ba-93f4-4dbc-ac39-4039fd254e7d\") " pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:48:53.723255 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:53.723217 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 03:48:53.723296 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:53.723278 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret podName:06a274ba-93f4-4dbc-ac39-4039fd254e7d nodeName:}" failed. No retries permitted until 2026-11-04 03:48:57.723258691 +0000 UTC m=+10.092719353 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret") pod "global-pull-secret-syncer-p84rx" (UID: "06a274ba-93f4-4dbc-ac39-4039fd254e7d") : object "kube-system"/"original-pull-secret" not registered Nov 04 03:48:53.824706 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:53.824061 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lvzv\" (UniqueName: \"kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv\") pod \"network-check-target-967tl\" (UID: \"5cf7e388-1678-4afb-8877-30179162401b\") " pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:48:53.824706 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:53.824237 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 03:48:53.824706 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:53.824254 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 03:48:53.824706 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:53.824265 2575 projected.go:196] Error preparing data for projected volume kube-api-access-8lvzv for pod openshift-network-diagnostics/network-check-target-967tl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 03:48:53.824706 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:53.824323 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv podName:5cf7e388-1678-4afb-8877-30179162401b nodeName:}" failed. No retries permitted until 2026-11-04 03:48:57.824302975 +0000 UTC m=+10.193763651 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8lvzv" (UniqueName: "kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv") pod "network-check-target-967tl" (UID: "5cf7e388-1678-4afb-8877-30179162401b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 03:48:54.167389 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:54.167350 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:48:54.168060 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:54.167502 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p84rx" podUID="06a274ba-93f4-4dbc-ac39-4039fd254e7d" Nov 04 03:48:54.168060 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:54.167922 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:48:54.168060 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:54.168041 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-967tl" podUID="5cf7e388-1678-4afb-8877-30179162401b" Nov 04 03:48:54.287880 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:54.287833 2575 generic.go:356] "Generic (PLEG): container finished" podID="295feeff6b2d9f63532642beaf61c650" containerID="231c4c085ba1655bdcc09e68b876000b1bf12834e03b7af23c19c813234e53a3" exitCode=0 Nov 04 03:48:54.288058 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:54.287961 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" event={"ID":"295feeff6b2d9f63532642beaf61c650","Type":"ContainerDied","Data":"231c4c085ba1655bdcc09e68b876000b1bf12834e03b7af23c19c813234e53a3"} Nov 04 03:48:54.294874 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:54.294450 2575 generic.go:356] "Generic (PLEG): container finished" podID="48e3e3bd-2ec1-4587-97b3-7b37fc5130fb" containerID="57f4fbd8debd9105bb20a0ff76091118622217dd8f52b734e646facdb49b3665" exitCode=0 Nov 04 03:48:54.295168 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:54.294865 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lrdgr" event={"ID":"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb","Type":"ContainerDied","Data":"57f4fbd8debd9105bb20a0ff76091118622217dd8f52b734e646facdb49b3665"} Nov 04 03:48:55.164255 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:55.164213 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:48:55.164453 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:55.164367 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p422k" podUID="56348938-ef44-41ab-8a71-acc1e4308262" Nov 04 03:48:55.301631 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:55.301590 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" event={"ID":"295feeff6b2d9f63532642beaf61c650","Type":"ContainerStarted","Data":"7d6b6808d6af8b504747ae2de12f868ec02b2799db72a975ef3b175911598126"} Nov 04 03:48:55.307069 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:55.305490 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-895vv" event={"ID":"dde9ca65-5089-450e-97cd-89bf66adba30","Type":"ContainerStarted","Data":"a31df1c12ca9d53804970b2eacba33e323ae4d5226f63e31de52cc8ec022940b"} Nov 04 03:48:55.343100 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:55.342363 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" podStartSLOduration=5.3423417539999996 podStartE2EDuration="5.342341754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:48:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:48:55.322342255 +0000 UTC m=+7.691802936" watchObservedRunningTime="2026-11-04 03:48:55.342341754 +0000 UTC m=+7.711802436" Nov 04 03:48:56.166821 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:56.166729 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:48:56.166995 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:56.166886 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-967tl" podUID="5cf7e388-1678-4afb-8877-30179162401b" Nov 04 03:48:56.167336 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:56.167311 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:48:56.167484 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:56.167431 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p84rx" podUID="06a274ba-93f4-4dbc-ac39-4039fd254e7d" Nov 04 03:48:56.311622 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:56.311586 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7vkwm" event={"ID":"e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8","Type":"ContainerStarted","Data":"11f624b421839a9dbe355254a6e800f752b1d4ed98a8c073f30cf5af503959ba"} Nov 04 03:48:56.330056 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:56.329990 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-895vv" podStartSLOduration=3.160212867 podStartE2EDuration="7.329968544s" totalImagesPullingTime="4.169755677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:48:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:48:55.342999737 +0000 UTC m=+7.712460419" watchObservedRunningTime="2026-11-04 03:48:56.329968544 +0000 UTC m=+8.699429226" Nov 04 03:48:56.330406 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:56.330366 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7vkwm" podStartSLOduration=1.8585487920000001 podStartE2EDuration="7.330356954s" totalImagesPullingTime="5.471808162s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:48:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:48:56.329524703 +0000 UTC m=+8.698985383" watchObservedRunningTime="2026-11-04 03:48:56.330356954 +0000 UTC m=+8.699817635" Nov 04 03:48:57.164820 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:57.164782 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:48:57.164989 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:57.164945 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p422k" podUID="56348938-ef44-41ab-8a71-acc1e4308262" Nov 04 03:48:57.317276 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:57.317239 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" event={"ID":"eaec1015-2b2b-4f06-b41d-028671919cf0","Type":"ContainerStarted","Data":"cb743cbfb4467e522cb9c7ed035a7955e421c77cd4631d60535848fb5d205581"} Nov 04 03:48:57.655737 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:57.655641 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs\") pod \"network-metrics-daemon-p422k\" (UID: \"56348938-ef44-41ab-8a71-acc1e4308262\") " pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:48:57.655977 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:57.655868 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 03:48:57.655977 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:57.655939 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs podName:56348938-ef44-41ab-8a71-acc1e4308262 nodeName:}" failed. No retries permitted until 2026-11-04 03:49:05.65592042 +0000 UTC m=+18.025381096 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs") pod "network-metrics-daemon-p422k" (UID: "56348938-ef44-41ab-8a71-acc1e4308262") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 03:48:57.756374 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:57.756323 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret\") pod \"global-pull-secret-syncer-p84rx\" (UID: \"06a274ba-93f4-4dbc-ac39-4039fd254e7d\") " pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:48:57.756563 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:57.756512 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 03:48:57.756626 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:57.756595 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret podName:06a274ba-93f4-4dbc-ac39-4039fd254e7d nodeName:}" failed. No retries permitted until 2026-11-04 03:49:05.756575583 +0000 UTC m=+18.126036254 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret") pod "global-pull-secret-syncer-p84rx" (UID: "06a274ba-93f4-4dbc-ac39-4039fd254e7d") : object "kube-system"/"original-pull-secret" not registered Nov 04 03:48:57.858157 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:57.858106 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lvzv\" (UniqueName: \"kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv\") pod \"network-check-target-967tl\" (UID: \"5cf7e388-1678-4afb-8877-30179162401b\") " pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:48:57.858356 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:57.858335 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 03:48:57.858432 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:57.858369 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 03:48:57.858432 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:57.858384 2575 projected.go:196] Error preparing data for projected volume kube-api-access-8lvzv for pod openshift-network-diagnostics/network-check-target-967tl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 03:48:57.858539 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:57.858452 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv podName:5cf7e388-1678-4afb-8877-30179162401b nodeName:}" failed. No retries permitted until 2026-11-04 03:49:05.858431089 +0000 UTC m=+18.227891773 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8lvzv" (UniqueName: "kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv") pod "network-check-target-967tl" (UID: "5cf7e388-1678-4afb-8877-30179162401b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 03:48:58.167303 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:58.167150 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:48:58.167303 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:58.167279 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-967tl" podUID="5cf7e388-1678-4afb-8877-30179162401b" Nov 04 03:48:58.167687 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:58.167665 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:48:58.167780 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:58.167760 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p84rx" podUID="06a274ba-93f4-4dbc-ac39-4039fd254e7d" Nov 04 03:48:58.398064 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:58.398005 2575 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-895vv" Nov 04 03:48:58.399654 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:58.399413 2575 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-895vv" Nov 04 03:48:58.786464 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:58.786132 2575 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Nov 04 03:48:59.050068 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:59.049933 2575 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T03:48:58.786404566Z","UUID":"877c802a-1559-455e-ae3c-9f12b220d5f5","Handler":null,"Name":"","Endpoint":""} Nov 04 03:48:59.052021 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:59.051992 2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Nov 04 03:48:59.052162 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:59.052039 2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Nov 04 03:48:59.164718 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:59.164683 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:48:59.164905 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:48:59.164826 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p422k" podUID="56348938-ef44-41ab-8a71-acc1e4308262" Nov 04 03:48:59.324444 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:59.324177 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" event={"ID":"eaec1015-2b2b-4f06-b41d-028671919cf0","Type":"ContainerStarted","Data":"b32c79948748e8c41aa3606bb0cc122072d4d726ba780fe227d3cf689611859e"} Nov 04 03:48:59.326805 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:59.326624 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5kmqc" event={"ID":"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07","Type":"ContainerStarted","Data":"eb61e667e36dd0bb86658880d581edb24585fef66be50bb31ccc85840552b58d"} Nov 04 03:48:59.329485 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:59.329452 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dxmg7" event={"ID":"2b70504f-4169-44b2-8b4e-979a068f070d","Type":"ContainerStarted","Data":"e1c578d4ace19f8a939dbc1d487c691ccbf2dde58de1169e393e5ab5307d5364"} Nov 04 03:48:59.371053 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:59.370988 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-dxmg7" podStartSLOduration=1.567807732 podStartE2EDuration="10.370971928s" totalImagesPullingTime="8.803164196s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:48:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:48:59.370501195 +0000 UTC m=+11.739961875" watchObservedRunningTime="2026-11-04 03:48:59.370971928 +0000 UTC m=+11.740432608" Nov 04 03:48:59.371246 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:48:59.371221 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-5kmqc" podStartSLOduration=2.056507351 podStartE2EDuration="10.371213702s" totalImagesPullingTime="8.314706351s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:48:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:48:59.350715845 +0000 UTC m=+11.720176525" watchObservedRunningTime="2026-11-04 03:48:59.371213702 +0000 UTC m=+11.740674381" Nov 04 03:49:00.164163 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:00.164136 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:49:00.164163 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:00.164171 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:49:00.165252 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:00.164251 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p84rx" podUID="06a274ba-93f4-4dbc-ac39-4039fd254e7d" Nov 04 03:49:00.165252 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:00.164410 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-967tl" podUID="5cf7e388-1678-4afb-8877-30179162401b" Nov 04 03:49:00.334101 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:00.334062 2575 generic.go:356] "Generic (PLEG): container finished" podID="48e3e3bd-2ec1-4587-97b3-7b37fc5130fb" containerID="12f37ca2f4f9e5f6f03dad61b237ada4791fb6f394515af5a20d5dee40f980c4" exitCode=0 Nov 04 03:49:00.334273 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:00.334159 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lrdgr" event={"ID":"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb","Type":"ContainerDied","Data":"12f37ca2f4f9e5f6f03dad61b237ada4791fb6f394515af5a20d5dee40f980c4"} Nov 04 03:49:00.336762 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:00.336733 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" event={"ID":"eaec1015-2b2b-4f06-b41d-028671919cf0","Type":"ContainerStarted","Data":"0c29e6230f5b7a2dffa8fc128e99c8951b7878f76962eccfea83e571a9a117de"} Nov 04 03:49:00.386080 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:00.386011 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" podStartSLOduration=2.127372028 podStartE2EDuration="11.385992065s" totalImagesPullingTime="9.258620037s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:48:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:00.385956745 +0000 UTC m=+12.755417427" watchObservedRunningTime="2026-11-04 03:49:00.385992065 +0000 UTC m=+12.755452747" Nov 04 03:49:01.165109 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:01.165074 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:49:01.165532 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:01.165239 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p422k" podUID="56348938-ef44-41ab-8a71-acc1e4308262" Nov 04 03:49:01.345765 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:01.345731 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mm4gp" event={"ID":"5bdb7796-b1a3-49a5-81cd-c8101320e172","Type":"ContainerStarted","Data":"5f39c775475ba39a10c7cb53cc6c8de7f906c24e2b2390f10765c467fd7c0ec9"} Nov 04 03:49:02.164599 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:02.164336 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:49:02.164779 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:02.164358 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:49:02.164779 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:02.164712 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-967tl" podUID="5cf7e388-1678-4afb-8877-30179162401b" Nov 04 03:49:02.164883 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:02.164810 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p84rx" podUID="06a274ba-93f4-4dbc-ac39-4039fd254e7d" Nov 04 03:49:02.349281 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:02.349236 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" event={"ID":"3e07cc7a-88c6-4e29-b7ab-c078916cd977","Type":"ContainerStarted","Data":"345559e37e502d266140e1a4405b480d2f6fecc78caa918d079ead2cb50cbc02"} Nov 04 03:49:02.351801 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:02.351770 2575 generic.go:356] "Generic (PLEG): container finished" podID="48e3e3bd-2ec1-4587-97b3-7b37fc5130fb" containerID="4a9c1ea6b00073ed4fc47deab87fc95db9f2f96d4b41a97debb593bb20942249" exitCode=0 Nov 04 03:49:02.351934 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:02.351863 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lrdgr" event={"ID":"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb","Type":"ContainerDied","Data":"4a9c1ea6b00073ed4fc47deab87fc95db9f2f96d4b41a97debb593bb20942249"} Nov 04 03:49:02.369504 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:02.369455 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" podStartSLOduration=2.461268977 podStartE2EDuration="13.369442468s" totalImagesPullingTime="10.908173491s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:48:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:02.369341223 +0000 UTC m=+14.738801899" watchObservedRunningTime="2026-11-04 03:49:02.369442468 +0000 UTC m=+14.738903146" Nov 04 03:49:02.369893 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:02.369840 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mm4gp" podStartSLOduration=4.111627694 podStartE2EDuration="13.369826651s" totalImagesPullingTime="9.258198957s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:48:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:01.364387939 +0000 UTC m=+13.733848619" watchObservedRunningTime="2026-11-04 03:49:02.369826651 +0000 UTC m=+14.739287329" Nov 04 03:49:03.164337 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:03.164299 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:49:03.164491 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:03.164467 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p422k" podUID="56348938-ef44-41ab-8a71-acc1e4308262" Nov 04 03:49:03.356482 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:03.356387 2575 generic.go:356] "Generic (PLEG): container finished" podID="48e3e3bd-2ec1-4587-97b3-7b37fc5130fb" containerID="aaa944c2a73d5beb680511d9984b12ee93bee1d618ac914fd12f9f2af8c96d7f" exitCode=0 Nov 04 03:49:03.356482 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:03.356469 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lrdgr" event={"ID":"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb","Type":"ContainerDied","Data":"aaa944c2a73d5beb680511d9984b12ee93bee1d618ac914fd12f9f2af8c96d7f"} Nov 04 03:49:04.166900 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:04.166868 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:49:04.167057 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:04.166991 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-967tl" podUID="5cf7e388-1678-4afb-8877-30179162401b" Nov 04 03:49:04.167379 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:04.167361 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:49:04.167475 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:04.167456 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p84rx" podUID="06a274ba-93f4-4dbc-ac39-4039fd254e7d" Nov 04 03:49:05.164815 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:05.164707 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:49:05.165187 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:05.164872 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p422k" podUID="56348938-ef44-41ab-8a71-acc1e4308262" Nov 04 03:49:05.364403 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:05.364328 2575 generic.go:356] "Generic (PLEG): container finished" podID="48e3e3bd-2ec1-4587-97b3-7b37fc5130fb" containerID="d3b5e94dafd63e4636c937a9a57887768ec5755e659e01828029c0282bd3fbdc" exitCode=0 Nov 04 03:49:05.364403 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:05.364375 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lrdgr" event={"ID":"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb","Type":"ContainerDied","Data":"d3b5e94dafd63e4636c937a9a57887768ec5755e659e01828029c0282bd3fbdc"} Nov 04 03:49:05.725437 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:05.725345 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs\") pod \"network-metrics-daemon-p422k\" (UID: \"56348938-ef44-41ab-8a71-acc1e4308262\") " pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:49:05.725585 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:05.725516 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 03:49:05.725624 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:05.725586 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs podName:56348938-ef44-41ab-8a71-acc1e4308262 nodeName:}" failed. No retries permitted until 2026-11-04 03:49:21.725572199 +0000 UTC m=+34.095032866 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs") pod "network-metrics-daemon-p422k" (UID: "56348938-ef44-41ab-8a71-acc1e4308262") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 03:49:05.825897 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:05.825857 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret\") pod \"global-pull-secret-syncer-p84rx\" (UID: \"06a274ba-93f4-4dbc-ac39-4039fd254e7d\") " pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:49:05.826086 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:05.826061 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 03:49:05.826155 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:05.826142 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret podName:06a274ba-93f4-4dbc-ac39-4039fd254e7d nodeName:}" failed. No retries permitted until 2026-11-04 03:49:21.826122037 +0000 UTC m=+34.195582694 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret") pod "global-pull-secret-syncer-p84rx" (UID: "06a274ba-93f4-4dbc-ac39-4039fd254e7d") : object "kube-system"/"original-pull-secret" not registered Nov 04 03:49:05.927378 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:05.927288 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lvzv\" (UniqueName: \"kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv\") pod \"network-check-target-967tl\" (UID: \"5cf7e388-1678-4afb-8877-30179162401b\") " pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:49:05.927529 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:05.927458 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 03:49:05.927529 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:05.927483 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 03:49:05.927529 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:05.927495 2575 projected.go:196] Error preparing data for projected volume kube-api-access-8lvzv for pod openshift-network-diagnostics/network-check-target-967tl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 03:49:05.927738 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:05.927553 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv podName:5cf7e388-1678-4afb-8877-30179162401b nodeName:}" failed. No retries permitted until 2026-11-04 03:49:21.927539731 +0000 UTC m=+34.297000389 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8lvzv" (UniqueName: "kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv") pod "network-check-target-967tl" (UID: "5cf7e388-1678-4afb-8877-30179162401b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 03:49:06.164635 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:06.164595 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:49:06.164828 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:06.164608 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:49:06.164828 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:06.164746 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p84rx" podUID="06a274ba-93f4-4dbc-ac39-4039fd254e7d" Nov 04 03:49:06.165302 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:06.164837 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-967tl" podUID="5cf7e388-1678-4afb-8877-30179162401b" Nov 04 03:49:06.370602 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:06.370566 2575 generic.go:356] "Generic (PLEG): container finished" podID="48e3e3bd-2ec1-4587-97b3-7b37fc5130fb" containerID="552419065ac4d7a094ea549294c5de9824019129aaa0ef78ac28931a84ceed4f" exitCode=0 Nov 04 03:49:06.370786 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:06.370626 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lrdgr" event={"ID":"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb","Type":"ContainerDied","Data":"552419065ac4d7a094ea549294c5de9824019129aaa0ef78ac28931a84ceed4f"} Nov 04 03:49:07.165007 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:07.164968 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:49:07.165405 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:07.165085 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p422k" podUID="56348938-ef44-41ab-8a71-acc1e4308262" Nov 04 03:49:07.376054 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:07.376020 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lrdgr" event={"ID":"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb","Type":"ContainerStarted","Data":"c613b3cb5321841dcd1db32b7ed598a43fce7033b1911364b22f20b2f27aa7f8"} Nov 04 03:49:07.403548 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:07.403485 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-lrdgr" podStartSLOduration=6.92508982 podStartE2EDuration="18.403467227s" totalImagesPullingTime="11.478377407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:48:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:07.403161586 +0000 UTC m=+19.772622264" watchObservedRunningTime="2026-11-04 03:49:07.403467227 +0000 UTC m=+19.772927904" Nov 04 03:49:08.112956 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:08.112768 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-895vv" Nov 04 03:49:08.113112 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:08.113070 2575 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 03:49:08.113529 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:08.113511 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-895vv" Nov 04 03:49:08.164992 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:08.164960 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:49:08.165111 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:08.165093 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:49:08.165111 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:08.165088 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p84rx" podUID="06a274ba-93f4-4dbc-ac39-4039fd254e7d" Nov 04 03:49:08.165673 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:08.165161 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-967tl" podUID="5cf7e388-1678-4afb-8877-30179162401b" Nov 04 03:49:08.381432 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:08.381355 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zfgzh_42510c3d-0283-4901-9d23-a84793598ac9/ovn-acl-logging/0.log" Nov 04 03:49:08.381710 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:08.381688 2575 generic.go:356] "Generic (PLEG): container finished" podID="42510c3d-0283-4901-9d23-a84793598ac9" containerID="0ada75f8142f065cef35019bcbcb2dee6a63cfe3a6e816dbec9405235c8064fd" exitCode=1 Nov 04 03:49:08.381799 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:08.381778 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" event={"ID":"42510c3d-0283-4901-9d23-a84793598ac9","Type":"ContainerStarted","Data":"0ccd87e1d184085e7ae17330088cba4452d33087604190a011022737fa0ecef7"} Nov 04 03:49:08.381878 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:08.381864 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" event={"ID":"42510c3d-0283-4901-9d23-a84793598ac9","Type":"ContainerStarted","Data":"73e74e410134261f0c1296967bac5eb4447e6e90ebf44b1aac99406d5ace0910"} Nov 04 03:49:08.381937 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:08.381924 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" event={"ID":"42510c3d-0283-4901-9d23-a84793598ac9","Type":"ContainerStarted","Data":"fdbb4052966b5e878ac7063432060109244bb227cd28d8d25f164f88d411ac67"} Nov 04 03:49:08.381994 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:08.381981 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" event={"ID":"42510c3d-0283-4901-9d23-a84793598ac9","Type":"ContainerStarted","Data":"530e4674bd19e59c728e6ee1d989be89110fdca52e7c4520bbfb5d7642caac80"} Nov 04 03:49:08.382044 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:08.382032 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" event={"ID":"42510c3d-0283-4901-9d23-a84793598ac9","Type":"ContainerDied","Data":"0ada75f8142f065cef35019bcbcb2dee6a63cfe3a6e816dbec9405235c8064fd"} Nov 04 03:49:08.382103 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:08.382090 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" event={"ID":"42510c3d-0283-4901-9d23-a84793598ac9","Type":"ContainerStarted","Data":"4e634a2ec3d6b272a21b0bb4dc0b8e720214faa35480376e0339e66c6b0eddbd"} Nov 04 03:49:09.164806 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:09.164743 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:49:09.164974 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:09.164898 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p422k" podUID="56348938-ef44-41ab-8a71-acc1e4308262" Nov 04 03:49:10.164740 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:10.164708 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:49:10.165294 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:10.164848 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-967tl" podUID="5cf7e388-1678-4afb-8877-30179162401b" Nov 04 03:49:10.165294 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:10.164882 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:49:10.165294 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:10.164932 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p84rx" podUID="06a274ba-93f4-4dbc-ac39-4039fd254e7d" Nov 04 03:49:10.388632 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:10.388606 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zfgzh_42510c3d-0283-4901-9d23-a84793598ac9/ovn-acl-logging/0.log" Nov 04 03:49:10.389035 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:10.389011 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" event={"ID":"42510c3d-0283-4901-9d23-a84793598ac9","Type":"ContainerStarted","Data":"85f3ac271f082f99a790cc9facf532141305c57fcb042a6c9c93d6371b78d443"} Nov 04 03:49:11.164334 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:11.164297 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:49:11.164561 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:11.164408 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p422k" podUID="56348938-ef44-41ab-8a71-acc1e4308262" Nov 04 03:49:12.166654 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:12.166586 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:49:12.166654 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:12.166614 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:49:12.167064 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:12.166708 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p84rx" podUID="06a274ba-93f4-4dbc-ac39-4039fd254e7d" Nov 04 03:49:12.167064 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:12.166821 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-967tl" podUID="5cf7e388-1678-4afb-8877-30179162401b" Nov 04 03:49:12.396741 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:12.396707 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zfgzh_42510c3d-0283-4901-9d23-a84793598ac9/ovn-acl-logging/0.log" Nov 04 03:49:12.397362 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:12.397154 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" event={"ID":"42510c3d-0283-4901-9d23-a84793598ac9","Type":"ContainerStarted","Data":"5260ab7a07d3bc70700f2f97edc5398a313f7dbb670151bd97f1faab77374c04"} Nov 04 03:49:12.397726 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:12.397706 2575 scope.go:122] "RemoveContainer" containerID="0ada75f8142f065cef35019bcbcb2dee6a63cfe3a6e816dbec9405235c8064fd" Nov 04 03:49:12.398005 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:12.397984 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:49:12.398207 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:12.398192 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:49:12.416211 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:12.416187 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:49:12.416335 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:12.416327 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:49:13.164928 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:13.164886 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:49:13.165107 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:13.165003 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p422k" podUID="56348938-ef44-41ab-8a71-acc1e4308262" Nov 04 03:49:13.402960 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:13.402934 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zfgzh_42510c3d-0283-4901-9d23-a84793598ac9/ovn-acl-logging/0.log" Nov 04 03:49:13.403369 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:13.403254 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" event={"ID":"42510c3d-0283-4901-9d23-a84793598ac9","Type":"ContainerStarted","Data":"e5da8e9652bd130ed07ad83e8cae12781fc84f3e46f29cf39351ad29e2a7a5e6"} Nov 04 03:49:13.403423 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:13.403379 2575 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 03:49:13.438617 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:13.438520 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" podStartSLOduration=7.371222898 podStartE2EDuration="24.438503112s" totalImagesPullingTime="17.067280214s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:48:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:13.437946073 +0000 UTC m=+25.807406749" watchObservedRunningTime="2026-11-04 03:49:13.438503112 +0000 UTC m=+25.807963787" Nov 04 03:49:13.850998 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:13.850975 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dxmg7_2b70504f-4169-44b2-8b4e-979a068f070d/dns-node-resolver/0.log" Nov 04 03:49:14.054476 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:14.054431 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p84rx"] Nov 04 03:49:14.054644 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:14.054584 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:49:14.054721 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:14.054692 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p84rx" podUID="06a274ba-93f4-4dbc-ac39-4039fd254e7d" Nov 04 03:49:14.055173 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:14.055150 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-967tl"] Nov 04 03:49:14.055299 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:14.055249 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:49:14.055366 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:14.055343 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-967tl" podUID="5cf7e388-1678-4afb-8877-30179162401b" Nov 04 03:49:14.055791 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:14.055771 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p422k"] Nov 04 03:49:14.055868 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:14.055843 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:49:14.055924 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:14.055910 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p422k" podUID="56348938-ef44-41ab-8a71-acc1e4308262" Nov 04 03:49:14.406022 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:14.405947 2575 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 03:49:14.613866 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:14.613841 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7vkwm_e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8/node-ca/0.log" Nov 04 03:49:16.164678 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:16.164642 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:49:16.165084 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:16.164655 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:49:16.165084 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:16.164807 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:49:16.165084 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:16.164806 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p422k" podUID="56348938-ef44-41ab-8a71-acc1e4308262" Nov 04 03:49:16.165084 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:16.164909 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p84rx" podUID="06a274ba-93f4-4dbc-ac39-4039fd254e7d" Nov 04 03:49:16.165084 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:16.164971 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-967tl" podUID="5cf7e388-1678-4afb-8877-30179162401b" Nov 04 03:49:16.990055 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:16.989965 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:49:18.167692 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:18.167499 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:49:18.168164 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:18.167553 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:49:18.168164 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:18.167789 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p84rx" podUID="06a274ba-93f4-4dbc-ac39-4039fd254e7d" Nov 04 03:49:18.168164 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:18.167553 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:49:18.168164 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:18.167850 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p422k" podUID="56348938-ef44-41ab-8a71-acc1e4308262" Nov 04 03:49:18.168164 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:18.167892 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-967tl" podUID="5cf7e388-1678-4afb-8877-30179162401b" Nov 04 03:49:20.165112 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:20.165069 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:49:20.165547 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:20.165080 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:49:20.165547 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:20.165207 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:49:20.165547 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:20.165208 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p84rx" podUID="06a274ba-93f4-4dbc-ac39-4039fd254e7d" Nov 04 03:49:20.165547 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:20.165322 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p422k" podUID="56348938-ef44-41ab-8a71-acc1e4308262" Nov 04 03:49:20.165547 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:20.165407 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-967tl" podUID="5cf7e388-1678-4afb-8877-30179162401b" Nov 04 03:49:21.734743 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:21.734703 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs\") pod \"network-metrics-daemon-p422k\" (UID: \"56348938-ef44-41ab-8a71-acc1e4308262\") " pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:49:21.735233 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:21.734850 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 03:49:21.735233 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:21.734906 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs podName:56348938-ef44-41ab-8a71-acc1e4308262 nodeName:}" failed. No retries permitted until 2026-11-04 03:49:53.734892608 +0000 UTC m=+66.104353262 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs") pod "network-metrics-daemon-p422k" (UID: "56348938-ef44-41ab-8a71-acc1e4308262") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 03:49:21.835507 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:21.835469 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret\") pod \"global-pull-secret-syncer-p84rx\" (UID: \"06a274ba-93f4-4dbc-ac39-4039fd254e7d\") " pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:49:21.835631 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:21.835581 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 03:49:21.835670 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:21.835632 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret podName:06a274ba-93f4-4dbc-ac39-4039fd254e7d nodeName:}" failed. No retries permitted until 2026-11-04 03:49:53.83561969 +0000 UTC m=+66.205080348 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret") pod "global-pull-secret-syncer-p84rx" (UID: "06a274ba-93f4-4dbc-ac39-4039fd254e7d") : object "kube-system"/"original-pull-secret" not registered Nov 04 03:49:21.936221 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:21.936183 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lvzv\" (UniqueName: \"kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv\") pod \"network-check-target-967tl\" (UID: \"5cf7e388-1678-4afb-8877-30179162401b\") " pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:49:21.936367 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:21.936338 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 03:49:21.936367 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:21.936361 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 03:49:21.936427 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:21.936371 2575 projected.go:196] Error preparing data for projected volume kube-api-access-8lvzv for pod openshift-network-diagnostics/network-check-target-967tl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 03:49:21.936427 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:21.936420 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv podName:5cf7e388-1678-4afb-8877-30179162401b nodeName:}" failed. No retries permitted until 2026-11-04 03:49:53.936407905 +0000 UTC m=+66.305868558 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8lvzv" (UniqueName: "kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv") pod "network-check-target-967tl" (UID: "5cf7e388-1678-4afb-8877-30179162401b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 03:49:22.164743 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:22.164708 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:49:22.164946 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:22.164836 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p84rx" podUID="06a274ba-93f4-4dbc-ac39-4039fd254e7d" Nov 04 03:49:22.164946 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:22.164909 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:49:22.164946 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:22.164941 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:49:22.165123 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:22.165034 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-967tl" podUID="5cf7e388-1678-4afb-8877-30179162401b" Nov 04 03:49:22.165177 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:22.165135 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p422k" podUID="56348938-ef44-41ab-8a71-acc1e4308262" Nov 04 03:49:24.164846 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:24.164809 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:49:24.164846 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:24.164833 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:49:24.165398 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:24.164934 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-967tl" podUID="5cf7e388-1678-4afb-8877-30179162401b" Nov 04 03:49:24.165398 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:24.164980 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p84rx" podUID="06a274ba-93f4-4dbc-ac39-4039fd254e7d" Nov 04 03:49:24.165398 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:24.165022 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:49:24.165398 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:24.165095 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p422k" podUID="56348938-ef44-41ab-8a71-acc1e4308262" Nov 04 03:49:26.164597 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:26.164561 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:49:26.165031 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:26.164566 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:49:26.165031 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:26.164679 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p422k" podUID="56348938-ef44-41ab-8a71-acc1e4308262" Nov 04 03:49:26.165031 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:26.164775 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-967tl" podUID="5cf7e388-1678-4afb-8877-30179162401b" Nov 04 03:49:26.165031 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:26.164577 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:49:26.165031 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:26.164870 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p84rx" podUID="06a274ba-93f4-4dbc-ac39-4039fd254e7d" Nov 04 03:49:28.165794 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:28.165738 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:49:28.166189 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:28.165897 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:49:28.166189 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:28.165903 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p422k" podUID="56348938-ef44-41ab-8a71-acc1e4308262" Nov 04 03:49:28.166189 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:28.166014 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-967tl" podUID="5cf7e388-1678-4afb-8877-30179162401b" Nov 04 03:49:28.166189 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:28.166083 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:49:28.166189 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:28.166138 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p84rx" podUID="06a274ba-93f4-4dbc-ac39-4039fd254e7d" Nov 04 03:49:30.164370 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:30.164332 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:49:30.164906 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:30.164334 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:49:30.164906 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:30.164446 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p422k" podUID="56348938-ef44-41ab-8a71-acc1e4308262" Nov 04 03:49:30.164906 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:30.164334 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:49:30.164906 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:30.164517 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p84rx" podUID="06a274ba-93f4-4dbc-ac39-4039fd254e7d" Nov 04 03:49:30.164906 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:30.164576 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-967tl" podUID="5cf7e388-1678-4afb-8877-30179162401b" Nov 04 03:49:32.164365 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:32.164321 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:49:32.164365 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:32.164348 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:49:32.164792 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:32.164370 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:49:32.164792 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:32.164459 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p422k" podUID="56348938-ef44-41ab-8a71-acc1e4308262" Nov 04 03:49:32.164792 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:32.164576 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p84rx" podUID="06a274ba-93f4-4dbc-ac39-4039fd254e7d" Nov 04 03:49:32.164792 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:32.164649 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-967tl" podUID="5cf7e388-1678-4afb-8877-30179162401b" Nov 04 03:49:34.167618 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:34.167585 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:49:34.168046 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:34.167674 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:49:34.168046 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:34.167685 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-967tl" podUID="5cf7e388-1678-4afb-8877-30179162401b" Nov 04 03:49:34.168046 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:34.167751 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p84rx" podUID="06a274ba-93f4-4dbc-ac39-4039fd254e7d" Nov 04 03:49:34.168046 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:34.167774 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:49:34.168046 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:34.167846 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p422k" podUID="56348938-ef44-41ab-8a71-acc1e4308262" Nov 04 03:49:36.164817 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.164782 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:49:36.165220 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.164782 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:49:36.165220 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:36.164888 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-967tl" podUID="5cf7e388-1678-4afb-8877-30179162401b" Nov 04 03:49:36.165220 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:36.165002 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p422k" podUID="56348938-ef44-41ab-8a71-acc1e4308262" Nov 04 03:49:36.165220 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.164788 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:49:36.165220 ip-10-0-129-134 kubenswrapper[2575]: E1104 03:49:36.165118 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p84rx" podUID="06a274ba-93f4-4dbc-ac39-4039fd254e7d" Nov 04 03:49:36.436363 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.436289 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-134.us-east-2.compute.internal" event="NodeReady" Nov 04 03:49:36.436508 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.436404 2575 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Nov 04 03:49:36.494410 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.494376 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-c425l"] Nov 04 03:49:36.496162 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.496144 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vcxpf"] Nov 04 03:49:36.496282 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.496265 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-c425l" Nov 04 03:49:36.498398 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.498375 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vcxpf" Nov 04 03:49:36.503165 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.503139 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Nov 04 03:49:36.503305 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.503237 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Nov 04 03:49:36.503305 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.503253 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-49k7r\"" Nov 04 03:49:36.503305 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.503301 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Nov 04 03:49:36.505433 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.505415 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Nov 04 03:49:36.506647 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.506625 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-c425l"] Nov 04 03:49:36.510741 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.510598 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vcxpf"] Nov 04 03:49:36.519857 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.519830 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rx686"] Nov 04 03:49:36.522389 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.522371 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:49:36.540253 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.540223 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rx686"] Nov 04 03:49:36.603987 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.603955 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Nov 04 03:49:36.604155 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.604024 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Nov 04 03:49:36.604155 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.604101 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Nov 04 03:49:36.604221 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.604199 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Nov 04 03:49:36.604221 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.604199 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-n475p\"" Nov 04 03:49:36.604285 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.604255 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Nov 04 03:49:36.604327 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.604310 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ccqtd\"" Nov 04 03:49:36.632620 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.632588 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7hkr\" (UniqueName: \"kubernetes.io/projected/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-kube-api-access-z7hkr\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:49:36.632679 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.632627 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62d4x\" (UniqueName: \"kubernetes.io/projected/94c06e7a-0510-4787-8b60-45b185300091-kube-api-access-62d4x\") pod \"dns-default-vcxpf\" (UID: \"94c06e7a-0510-4787-8b60-45b185300091\") " pod="openshift-dns/dns-default-vcxpf" Nov 04 03:49:36.632747 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.632729 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea2d0497-b79b-4640-b8ca-bd829b1f3dfa-cert\") pod \"ingress-canary-c425l\" (UID: \"ea2d0497-b79b-4640-b8ca-bd829b1f3dfa\") " pod="openshift-ingress-canary/ingress-canary-c425l" Nov 04 03:49:36.632807 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.632782 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/94c06e7a-0510-4787-8b60-45b185300091-config-volume\") pod \"dns-default-vcxpf\" (UID: \"94c06e7a-0510-4787-8b60-45b185300091\") " pod="openshift-dns/dns-default-vcxpf" Nov 04 03:49:36.632871 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.632852 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-crio-socket\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:49:36.632904 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.632882 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:49:36.632938 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.632910 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/94c06e7a-0510-4787-8b60-45b185300091-metrics-tls\") pod \"dns-default-vcxpf\" (UID: \"94c06e7a-0510-4787-8b60-45b185300091\") " pod="openshift-dns/dns-default-vcxpf" Nov 04 03:49:36.632971 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.632950 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:49:36.633002 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.632990 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm29k\" (UniqueName: \"kubernetes.io/projected/ea2d0497-b79b-4640-b8ca-bd829b1f3dfa-kube-api-access-jm29k\") pod \"ingress-canary-c425l\" (UID: \"ea2d0497-b79b-4640-b8ca-bd829b1f3dfa\") " pod="openshift-ingress-canary/ingress-canary-c425l" Nov 04 03:49:36.633030 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.633011 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94c06e7a-0510-4787-8b60-45b185300091-tmp-dir\") pod \"dns-default-vcxpf\" (UID: \"94c06e7a-0510-4787-8b60-45b185300091\") " pod="openshift-dns/dns-default-vcxpf" Nov 04 03:49:36.633030 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.633027 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-data-volume\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:49:36.733563 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.733480 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/94c06e7a-0510-4787-8b60-45b185300091-metrics-tls\") pod \"dns-default-vcxpf\" (UID: \"94c06e7a-0510-4787-8b60-45b185300091\") " pod="openshift-dns/dns-default-vcxpf" Nov 04 03:49:36.733563 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.733519 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:49:36.733563 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.733541 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jm29k\" (UniqueName: \"kubernetes.io/projected/ea2d0497-b79b-4640-b8ca-bd829b1f3dfa-kube-api-access-jm29k\") pod \"ingress-canary-c425l\" (UID: \"ea2d0497-b79b-4640-b8ca-bd829b1f3dfa\") " pod="openshift-ingress-canary/ingress-canary-c425l" Nov 04 03:49:36.733818 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.733659 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94c06e7a-0510-4787-8b60-45b185300091-tmp-dir\") pod \"dns-default-vcxpf\" (UID: \"94c06e7a-0510-4787-8b60-45b185300091\") " pod="openshift-dns/dns-default-vcxpf" Nov 04 03:49:36.733818 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.733692 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-data-volume\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:49:36.733818 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.733714 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7hkr\" (UniqueName: \"kubernetes.io/projected/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-kube-api-access-z7hkr\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:49:36.733967 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.733947 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62d4x\" (UniqueName: \"kubernetes.io/projected/94c06e7a-0510-4787-8b60-45b185300091-kube-api-access-62d4x\") pod \"dns-default-vcxpf\" (UID: \"94c06e7a-0510-4787-8b60-45b185300091\") " pod="openshift-dns/dns-default-vcxpf" Nov 04 03:49:36.734041 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.734027 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea2d0497-b79b-4640-b8ca-bd829b1f3dfa-cert\") pod \"ingress-canary-c425l\" (UID: \"ea2d0497-b79b-4640-b8ca-bd829b1f3dfa\") " pod="openshift-ingress-canary/ingress-canary-c425l" Nov 04 03:49:36.734094 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.734056 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-data-volume\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:49:36.734094 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.734068 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/94c06e7a-0510-4787-8b60-45b185300091-config-volume\") pod \"dns-default-vcxpf\" (UID: \"94c06e7a-0510-4787-8b60-45b185300091\") " pod="openshift-dns/dns-default-vcxpf" Nov 04 03:49:36.734188 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.734097 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-crio-socket\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:49:36.734188 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.734137 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94c06e7a-0510-4787-8b60-45b185300091-tmp-dir\") pod \"dns-default-vcxpf\" (UID: \"94c06e7a-0510-4787-8b60-45b185300091\") " pod="openshift-dns/dns-default-vcxpf" Nov 04 03:49:36.734188 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.734176 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:49:36.734188 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.734180 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:49:36.734374 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.734263 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-crio-socket\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:49:36.734589 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.734567 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/94c06e7a-0510-4787-8b60-45b185300091-config-volume\") pod \"dns-default-vcxpf\" (UID: \"94c06e7a-0510-4787-8b60-45b185300091\") " pod="openshift-dns/dns-default-vcxpf" Nov 04 03:49:36.737537 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.737514 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:49:36.737638 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.737543 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/94c06e7a-0510-4787-8b60-45b185300091-metrics-tls\") pod \"dns-default-vcxpf\" (UID: \"94c06e7a-0510-4787-8b60-45b185300091\") " pod="openshift-dns/dns-default-vcxpf" Nov 04 03:49:36.737683 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.737667 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea2d0497-b79b-4640-b8ca-bd829b1f3dfa-cert\") pod \"ingress-canary-c425l\" (UID: \"ea2d0497-b79b-4640-b8ca-bd829b1f3dfa\") " pod="openshift-ingress-canary/ingress-canary-c425l" Nov 04 03:49:36.751162 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.751140 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm29k\" (UniqueName: \"kubernetes.io/projected/ea2d0497-b79b-4640-b8ca-bd829b1f3dfa-kube-api-access-jm29k\") pod \"ingress-canary-c425l\" (UID: \"ea2d0497-b79b-4640-b8ca-bd829b1f3dfa\") " pod="openshift-ingress-canary/ingress-canary-c425l" Nov 04 03:49:36.752273 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.752255 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62d4x\" (UniqueName: \"kubernetes.io/projected/94c06e7a-0510-4787-8b60-45b185300091-kube-api-access-62d4x\") pod \"dns-default-vcxpf\" (UID: \"94c06e7a-0510-4787-8b60-45b185300091\") " pod="openshift-dns/dns-default-vcxpf" Nov 04 03:49:36.756412 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.756392 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7hkr\" (UniqueName: \"kubernetes.io/projected/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-kube-api-access-z7hkr\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:49:36.810399 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.810361 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-c425l" Nov 04 03:49:36.817045 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.817020 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vcxpf" Nov 04 03:49:36.835157 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:36.835128 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:49:37.098876 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:37.098842 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-c425l"] Nov 04 03:49:37.101589 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:37.101559 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vcxpf"] Nov 04 03:49:37.103417 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:49:37.103387 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea2d0497_b79b_4640_b8ca_bd829b1f3dfa.slice/crio-7d0369f4f9735d20f2cb7c87a7d4aa713189f161cfb549cfb9b55f700873604c WatchSource:0}: Error finding container 7d0369f4f9735d20f2cb7c87a7d4aa713189f161cfb549cfb9b55f700873604c: Status 404 returned error can't find the container with id 7d0369f4f9735d20f2cb7c87a7d4aa713189f161cfb549cfb9b55f700873604c Nov 04 03:49:37.104743 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:49:37.104715 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94c06e7a_0510_4787_8b60_45b185300091.slice/crio-227320894eae3ad9b870f9978853be7605090c7c784e2642a0c239cc0ed64c68 WatchSource:0}: Error finding container 227320894eae3ad9b870f9978853be7605090c7c784e2642a0c239cc0ed64c68: Status 404 returned error can't find the container with id 227320894eae3ad9b870f9978853be7605090c7c784e2642a0c239cc0ed64c68 Nov 04 03:49:37.125365 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:37.125338 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rx686"] Nov 04 03:49:37.128228 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:49:37.128201 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeff746dd_87c2_47fb_bee2_f6ebb6c11fc6.slice/crio-23c9921b11f6c392fc6c134e044f2b9c16296d9fea93e4c7e98458ca442cdd8b WatchSource:0}: Error finding container 23c9921b11f6c392fc6c134e044f2b9c16296d9fea93e4c7e98458ca442cdd8b: Status 404 returned error can't find the container with id 23c9921b11f6c392fc6c134e044f2b9c16296d9fea93e4c7e98458ca442cdd8b Nov 04 03:49:37.456419 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:37.456334 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c425l" event={"ID":"ea2d0497-b79b-4640-b8ca-bd829b1f3dfa","Type":"ContainerStarted","Data":"7d0369f4f9735d20f2cb7c87a7d4aa713189f161cfb549cfb9b55f700873604c"} Nov 04 03:49:37.457732 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:37.457708 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rx686" event={"ID":"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6","Type":"ContainerStarted","Data":"a09443522a00978971540d1455e8054f58c6fa5663cf692278c206fab5f08009"} Nov 04 03:49:37.457867 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:37.457774 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rx686" event={"ID":"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6","Type":"ContainerStarted","Data":"23c9921b11f6c392fc6c134e044f2b9c16296d9fea93e4c7e98458ca442cdd8b"} Nov 04 03:49:37.458739 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:37.458719 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vcxpf" event={"ID":"94c06e7a-0510-4787-8b60-45b185300091","Type":"ContainerStarted","Data":"227320894eae3ad9b870f9978853be7605090c7c784e2642a0c239cc0ed64c68"} Nov 04 03:49:38.165839 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:38.165803 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:49:38.166039 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:38.165887 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:49:38.166112 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:38.166076 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:49:38.170137 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:38.170111 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 04 03:49:38.171357 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:38.171337 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 04 03:49:38.171863 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:38.171741 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 04 03:49:38.171863 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:38.171811 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-49g8t\"" Nov 04 03:49:38.171863 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:38.171819 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-p2j7s\"" Nov 04 03:49:38.171863 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:38.171825 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 04 03:49:38.171863 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:38.171863 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fmfbg\"" Nov 04 03:49:38.255290 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:38.255215 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dxmg7_2b70504f-4169-44b2-8b4e-979a068f070d/dns-node-resolver/0.log" Nov 04 03:49:38.462891 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:38.462849 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rx686" event={"ID":"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6","Type":"ContainerStarted","Data":"249bd669acfeb642b427b3d2baff181bde1790c619d71c77c74eb712e235373a"} Nov 04 03:49:39.256393 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:39.256366 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7vkwm_e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8/node-ca/0.log" Nov 04 03:49:39.723510 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:39.723478 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p"] Nov 04 03:49:39.736496 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:39.736460 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" Nov 04 03:49:39.739152 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:39.739122 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p"] Nov 04 03:49:39.746176 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:39.746060 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Nov 04 03:49:39.746176 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:39.746103 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Nov 04 03:49:39.746176 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:39.746134 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-s9kfd\"" Nov 04 03:49:39.746745 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:39.746721 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Nov 04 03:49:39.747088 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:39.747057 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Nov 04 03:49:39.747209 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:39.747190 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Nov 04 03:49:39.859703 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:39.859674 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2502af7a-1118-4c5e-84b8-8c97188c09a4-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-tnr8p\" (UID: \"2502af7a-1118-4c5e-84b8-8c97188c09a4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" Nov 04 03:49:39.859859 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:39.859711 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv8hw\" (UniqueName: \"kubernetes.io/projected/2502af7a-1118-4c5e-84b8-8c97188c09a4-kube-api-access-bv8hw\") pod \"prometheus-operator-9cd89b74c-tnr8p\" (UID: \"2502af7a-1118-4c5e-84b8-8c97188c09a4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" Nov 04 03:49:39.859914 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:39.859867 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2502af7a-1118-4c5e-84b8-8c97188c09a4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-tnr8p\" (UID: \"2502af7a-1118-4c5e-84b8-8c97188c09a4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" Nov 04 03:49:39.859914 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:39.859905 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2502af7a-1118-4c5e-84b8-8c97188c09a4-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-tnr8p\" (UID: \"2502af7a-1118-4c5e-84b8-8c97188c09a4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" Nov 04 03:49:39.960982 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:39.960942 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv8hw\" (UniqueName: \"kubernetes.io/projected/2502af7a-1118-4c5e-84b8-8c97188c09a4-kube-api-access-bv8hw\") pod \"prometheus-operator-9cd89b74c-tnr8p\" (UID: \"2502af7a-1118-4c5e-84b8-8c97188c09a4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" Nov 04 03:49:39.961151 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:39.961048 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2502af7a-1118-4c5e-84b8-8c97188c09a4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-tnr8p\" (UID: \"2502af7a-1118-4c5e-84b8-8c97188c09a4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" Nov 04 03:49:39.961151 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:39.961087 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2502af7a-1118-4c5e-84b8-8c97188c09a4-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-tnr8p\" (UID: \"2502af7a-1118-4c5e-84b8-8c97188c09a4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" Nov 04 03:49:39.961151 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:39.961114 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2502af7a-1118-4c5e-84b8-8c97188c09a4-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-tnr8p\" (UID: \"2502af7a-1118-4c5e-84b8-8c97188c09a4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" Nov 04 03:49:39.962293 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:39.962211 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2502af7a-1118-4c5e-84b8-8c97188c09a4-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-tnr8p\" (UID: \"2502af7a-1118-4c5e-84b8-8c97188c09a4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" Nov 04 03:49:39.963963 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:39.963942 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2502af7a-1118-4c5e-84b8-8c97188c09a4-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-tnr8p\" (UID: \"2502af7a-1118-4c5e-84b8-8c97188c09a4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" Nov 04 03:49:39.964112 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:39.964085 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2502af7a-1118-4c5e-84b8-8c97188c09a4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-tnr8p\" (UID: \"2502af7a-1118-4c5e-84b8-8c97188c09a4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" Nov 04 03:49:39.974595 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:39.974525 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv8hw\" (UniqueName: \"kubernetes.io/projected/2502af7a-1118-4c5e-84b8-8c97188c09a4-kube-api-access-bv8hw\") pod \"prometheus-operator-9cd89b74c-tnr8p\" (UID: \"2502af7a-1118-4c5e-84b8-8c97188c09a4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" Nov 04 03:49:40.083317 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:40.083280 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" Nov 04 03:49:40.377591 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:40.377552 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p"] Nov 04 03:49:40.383825 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:49:40.383743 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2502af7a_1118_4c5e_84b8_8c97188c09a4.slice/crio-e6265225ad2d7fd61a17e7506274cd6ee39345729f55043c8c0d1796c9f1d66a WatchSource:0}: Error finding container e6265225ad2d7fd61a17e7506274cd6ee39345729f55043c8c0d1796c9f1d66a: Status 404 returned error can't find the container with id e6265225ad2d7fd61a17e7506274cd6ee39345729f55043c8c0d1796c9f1d66a Nov 04 03:49:40.471733 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:40.471689 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rx686" event={"ID":"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6","Type":"ContainerStarted","Data":"bb2b4443e7de3d34696065c0c7ed98e69e334d1665964fa6d71bab9d1999e130"} Nov 04 03:49:40.473388 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:40.473363 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" event={"ID":"2502af7a-1118-4c5e-84b8-8c97188c09a4","Type":"ContainerStarted","Data":"e6265225ad2d7fd61a17e7506274cd6ee39345729f55043c8c0d1796c9f1d66a"} Nov 04 03:49:40.495041 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:40.494971 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rx686" podStartSLOduration=1.8436533800000001 podStartE2EDuration="4.494952744s" totalImagesPullingTime="2.651299364s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:49:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:40.492851806 +0000 UTC m=+52.862312477" watchObservedRunningTime="2026-11-04 03:49:40.494952744 +0000 UTC m=+52.864413423" Nov 04 03:49:41.479240 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:41.479204 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vcxpf" event={"ID":"94c06e7a-0510-4787-8b60-45b185300091","Type":"ContainerStarted","Data":"32a4a777ccbda019751cbeec405e6f17d1088ee410703de4100bbc80ad782f55"} Nov 04 03:49:41.484151 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:41.484121 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c425l" event={"ID":"ea2d0497-b79b-4640-b8ca-bd829b1f3dfa","Type":"ContainerStarted","Data":"fd4a4eb5cf307202bc3345d10ea8fafc93bc4ad3ab056aac6ce306a80ed056fd"} Nov 04 03:49:41.504287 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:41.504240 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-c425l" podStartSLOduration=1.299197045 podStartE2EDuration="5.50422494s" totalImagesPullingTime="4.205027895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:49:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:41.503444934 +0000 UTC m=+53.872905609" watchObservedRunningTime="2026-11-04 03:49:41.50422494 +0000 UTC m=+53.873685616" Nov 04 03:49:42.488856 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:42.488807 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vcxpf" event={"ID":"94c06e7a-0510-4787-8b60-45b185300091","Type":"ContainerStarted","Data":"b71a862e7edcd3cfa8dc37b5efeef3544f472b15b112bbbf54844264e94fb23c"} Nov 04 03:49:42.520074 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:42.520024 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-vcxpf" podStartSLOduration=2.32616341 podStartE2EDuration="6.520010251s" totalImagesPullingTime="4.193846841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:49:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:42.518136596 +0000 UTC m=+54.887597272" watchObservedRunningTime="2026-11-04 03:49:42.520010251 +0000 UTC m=+54.889470940" Nov 04 03:49:43.496599 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:43.496564 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" event={"ID":"2502af7a-1118-4c5e-84b8-8c97188c09a4","Type":"ContainerStarted","Data":"cc2a50a040777af78cd2ae00f260c2da76c87f2389ed5acb6ae95bef8e42dca8"} Nov 04 03:49:43.497094 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:43.496624 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" event={"ID":"2502af7a-1118-4c5e-84b8-8c97188c09a4","Type":"ContainerStarted","Data":"23313be1eab99e9c50e328511fd097e81e3645d221500e392430d13475c308d1"} Nov 04 03:49:43.497094 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:43.496868 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vcxpf" Nov 04 03:49:43.535891 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:43.535828 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" podStartSLOduration=2.126478487 podStartE2EDuration="4.535809346s" totalImagesPullingTime="2.409330859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:49:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:43.534924482 +0000 UTC m=+55.904385158" watchObservedRunningTime="2026-11-04 03:49:43.535809346 +0000 UTC m=+55.905270022" Nov 04 03:49:45.173193 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.173160 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-r7srf"] Nov 04 03:49:45.175658 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.175637 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.228963 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.228930 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rh4kv\"" Nov 04 03:49:45.228963 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.228947 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Nov 04 03:49:45.229252 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.229238 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Nov 04 03:49:45.254832 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.254797 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Nov 04 03:49:45.302111 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.302069 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b1091f60-668b-4884-8548-fe4f4c438367-sys\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.302111 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.302113 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b1091f60-668b-4884-8548-fe4f4c438367-root\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.302320 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.302133 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1091f60-668b-4884-8548-fe4f4c438367-metrics-client-ca\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.302320 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.302155 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.302320 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.302301 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-textfile\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.302425 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.302332 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-wtmp\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.302425 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.302349 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-accelerators-collector-config\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.302425 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.302367 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgmb8\" (UniqueName: \"kubernetes.io/projected/b1091f60-668b-4884-8548-fe4f4c438367-kube-api-access-bgmb8\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.302519 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.302430 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-tls\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.402879 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.402835 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-tls\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.402879 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.402878 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b1091f60-668b-4884-8548-fe4f4c438367-sys\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.403085 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.402902 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b1091f60-668b-4884-8548-fe4f4c438367-root\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.403085 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.402920 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1091f60-668b-4884-8548-fe4f4c438367-metrics-client-ca\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.403085 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.402947 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.403085 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.402971 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b1091f60-668b-4884-8548-fe4f4c438367-sys\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.403085 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.402981 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b1091f60-668b-4884-8548-fe4f4c438367-root\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.403085 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.403055 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-textfile\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.403085 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.403074 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-wtmp\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.403338 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.403093 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-accelerators-collector-config\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.403338 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.403111 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgmb8\" (UniqueName: \"kubernetes.io/projected/b1091f60-668b-4884-8548-fe4f4c438367-kube-api-access-bgmb8\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.403338 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.403247 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-wtmp\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.403542 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.403519 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-textfile\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.403619 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.403565 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1091f60-668b-4884-8548-fe4f4c438367-metrics-client-ca\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.403659 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.403635 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-accelerators-collector-config\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.405813 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.405790 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-tls\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.405915 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.405828 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.419571 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.419541 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgmb8\" (UniqueName: \"kubernetes.io/projected/b1091f60-668b-4884-8548-fe4f4c438367-kube-api-access-bgmb8\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.486979 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.486890 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:49:45.496015 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:49:45.495987 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1091f60_668b_4884_8548_fe4f4c438367.slice/crio-5e81bd6796820adb866f5400e7a2075890b09341253469f90fa40cac9a7346be WatchSource:0}: Error finding container 5e81bd6796820adb866f5400e7a2075890b09341253469f90fa40cac9a7346be: Status 404 returned error can't find the container with id 5e81bd6796820adb866f5400e7a2075890b09341253469f90fa40cac9a7346be Nov 04 03:49:45.503629 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:45.503602 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r7srf" event={"ID":"b1091f60-668b-4884-8548-fe4f4c438367","Type":"ContainerStarted","Data":"5e81bd6796820adb866f5400e7a2075890b09341253469f90fa40cac9a7346be"} Nov 04 03:49:47.001585 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.001556 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:49:47.217138 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.217053 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7976867cc6-jgp9n"] Nov 04 03:49:47.222062 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.222035 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.226840 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.226792 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Nov 04 03:49:47.226995 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.226842 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Nov 04 03:49:47.226995 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.226941 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-7cu829nqfpjf\"" Nov 04 03:49:47.226995 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.226944 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Nov 04 03:49:47.226995 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.226967 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Nov 04 03:49:47.226995 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.226972 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Nov 04 03:49:47.227177 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.227023 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-qc8ts\"" Nov 04 03:49:47.235878 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.235853 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7976867cc6-jgp9n"] Nov 04 03:49:47.317379 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.317342 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a6918a9-3ff2-46aa-bc91-affd1835403d-tmp\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.317379 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.317383 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.317587 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.317405 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.317587 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.317491 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-tls\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.317587 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.317539 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjqhd\" (UniqueName: \"kubernetes.io/projected/8a6918a9-3ff2-46aa-bc91-affd1835403d-kube-api-access-hjqhd\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.317690 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.317627 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.317690 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.317656 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a6918a9-3ff2-46aa-bc91-affd1835403d-metrics-client-ca\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.317792 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.317691 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-grpc-tls\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.317792 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.317710 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.418022 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.417987 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.418022 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.418027 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a6918a9-3ff2-46aa-bc91-affd1835403d-metrics-client-ca\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.418192 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.418049 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-grpc-tls\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.418192 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.418070 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.418192 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.418135 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a6918a9-3ff2-46aa-bc91-affd1835403d-tmp\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.418192 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.418167 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.418386 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.418199 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.418386 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.418225 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-tls\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.418386 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.418249 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjqhd\" (UniqueName: \"kubernetes.io/projected/8a6918a9-3ff2-46aa-bc91-affd1835403d-kube-api-access-hjqhd\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.418723 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.418697 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a6918a9-3ff2-46aa-bc91-affd1835403d-tmp\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.418872 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.418799 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a6918a9-3ff2-46aa-bc91-affd1835403d-metrics-client-ca\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.420821 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.420797 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.420944 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.420922 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-grpc-tls\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.421107 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.421081 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.421189 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.421086 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.421266 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.421244 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-tls\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.421316 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.421283 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.431921 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.431896 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjqhd\" (UniqueName: \"kubernetes.io/projected/8a6918a9-3ff2-46aa-bc91-affd1835403d-kube-api-access-hjqhd\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.512326 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.512237 2575 generic.go:356] "Generic (PLEG): container finished" podID="b1091f60-668b-4884-8548-fe4f4c438367" containerID="96557d6e4cf1cfe4b85971ecdcff920b326b464f7cf8f30a4edb83b2b1d338e1" exitCode=0 Nov 04 03:49:47.512473 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.512321 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r7srf" event={"ID":"b1091f60-668b-4884-8548-fe4f4c438367","Type":"ContainerDied","Data":"96557d6e4cf1cfe4b85971ecdcff920b326b464f7cf8f30a4edb83b2b1d338e1"} Nov 04 03:49:47.536527 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.536499 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:47.825000 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:47.824963 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7976867cc6-jgp9n"] Nov 04 03:49:47.829048 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:49:47.829018 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a6918a9_3ff2_46aa_bc91_affd1835403d.slice/crio-97588462a6ec62deb01f9d16792c6618b07ca79108ede868df2fb0c74bff5a84 WatchSource:0}: Error finding container 97588462a6ec62deb01f9d16792c6618b07ca79108ede868df2fb0c74bff5a84: Status 404 returned error can't find the container with id 97588462a6ec62deb01f9d16792c6618b07ca79108ede868df2fb0c74bff5a84 Nov 04 03:49:48.518124 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:48.518075 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r7srf" event={"ID":"b1091f60-668b-4884-8548-fe4f4c438367","Type":"ContainerStarted","Data":"69797a05d179cd19dcc5f7bd0f0350df7b2ce583228023620b1210609c913720"} Nov 04 03:49:48.518587 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:48.518149 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r7srf" event={"ID":"b1091f60-668b-4884-8548-fe4f4c438367","Type":"ContainerStarted","Data":"f0ee65dba05e0965a265014ce5eba8eaa0379c3b873d7b70ed311f6cd8a6ba0e"} Nov 04 03:49:48.519422 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:48.519390 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" event={"ID":"8a6918a9-3ff2-46aa-bc91-affd1835403d","Type":"ContainerStarted","Data":"97588462a6ec62deb01f9d16792c6618b07ca79108ede868df2fb0c74bff5a84"} Nov 04 03:49:48.542003 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:48.541943 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-r7srf" podStartSLOduration=2.081039802 podStartE2EDuration="3.541921116s" totalImagesPullingTime="1.384720551s" totalInitContainerRuntime="76.160763ms" isStatefulPod=true podCreationTimestamp="2026-11-04 03:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:48.540945724 +0000 UTC m=+60.910406414" watchObservedRunningTime="2026-11-04 03:49:48.541921116 +0000 UTC m=+60.911381791" Nov 04 03:49:49.889697 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:49.889660 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-f92gv"] Nov 04 03:49:49.892622 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:49.892602 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-f92gv" Nov 04 03:49:49.897168 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:49.897141 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Nov 04 03:49:49.897168 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:49.897148 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-d55th\"" Nov 04 03:49:49.905824 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:49.905793 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-f92gv"] Nov 04 03:49:49.941156 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:49.941118 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf87g\" (UniqueName: \"kubernetes.io/projected/27390c92-0480-4b55-ba3d-e20f7ada5b83-kube-api-access-gf87g\") pod \"monitoring-plugin-77f4f76594-f92gv\" (UID: \"27390c92-0480-4b55-ba3d-e20f7ada5b83\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-f92gv" Nov 04 03:49:49.941333 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:49.941203 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/27390c92-0480-4b55-ba3d-e20f7ada5b83-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-f92gv\" (UID: \"27390c92-0480-4b55-ba3d-e20f7ada5b83\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-f92gv" Nov 04 03:49:50.042214 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:50.042171 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/27390c92-0480-4b55-ba3d-e20f7ada5b83-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-f92gv\" (UID: \"27390c92-0480-4b55-ba3d-e20f7ada5b83\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-f92gv" Nov 04 03:49:50.042420 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:50.042378 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf87g\" (UniqueName: \"kubernetes.io/projected/27390c92-0480-4b55-ba3d-e20f7ada5b83-kube-api-access-gf87g\") pod \"monitoring-plugin-77f4f76594-f92gv\" (UID: \"27390c92-0480-4b55-ba3d-e20f7ada5b83\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-f92gv" Nov 04 03:49:50.045234 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:50.045207 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/27390c92-0480-4b55-ba3d-e20f7ada5b83-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-f92gv\" (UID: \"27390c92-0480-4b55-ba3d-e20f7ada5b83\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-f92gv" Nov 04 03:49:50.055511 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:50.055480 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf87g\" (UniqueName: \"kubernetes.io/projected/27390c92-0480-4b55-ba3d-e20f7ada5b83-kube-api-access-gf87g\") pod \"monitoring-plugin-77f4f76594-f92gv\" (UID: \"27390c92-0480-4b55-ba3d-e20f7ada5b83\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-f92gv" Nov 04 03:49:50.209063 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:50.207725 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-f92gv" Nov 04 03:49:50.499463 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:50.499371 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-f92gv"] Nov 04 03:49:50.502562 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:49:50.502521 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27390c92_0480_4b55_ba3d_e20f7ada5b83.slice/crio-bbf6bd95a7b883a52463bb9f0aa2c253329afd5c317a40ec8cb5bdf55a17c43f WatchSource:0}: Error finding container bbf6bd95a7b883a52463bb9f0aa2c253329afd5c317a40ec8cb5bdf55a17c43f: Status 404 returned error can't find the container with id bbf6bd95a7b883a52463bb9f0aa2c253329afd5c317a40ec8cb5bdf55a17c43f Nov 04 03:49:50.527180 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:50.527143 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-f92gv" event={"ID":"27390c92-0480-4b55-ba3d-e20f7ada5b83","Type":"ContainerStarted","Data":"bbf6bd95a7b883a52463bb9f0aa2c253329afd5c317a40ec8cb5bdf55a17c43f"} Nov 04 03:49:50.529220 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:50.529186 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" event={"ID":"8a6918a9-3ff2-46aa-bc91-affd1835403d","Type":"ContainerStarted","Data":"634eef75252a3f62e92b16abda4ca573b895b20157fa5e1284525f0f292be0c4"} Nov 04 03:49:50.529324 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:50.529249 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" event={"ID":"8a6918a9-3ff2-46aa-bc91-affd1835403d","Type":"ContainerStarted","Data":"02a1d982d7bfdd2e854758d0dcad9c318a38a7db1e2e6c44d475d7c5d3c4ca46"} Nov 04 03:49:50.529324 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:50.529274 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" event={"ID":"8a6918a9-3ff2-46aa-bc91-affd1835403d","Type":"ContainerStarted","Data":"365c409ea7d34f1ee3c68452375a6f639c364e3d0ebda07c72fac2812b8464a3"} Nov 04 03:49:51.429479 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.429302 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 03:49:51.433352 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.433304 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.446375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.446112 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Nov 04 03:49:51.446375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.446173 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Nov 04 03:49:51.446375 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.446189 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Nov 04 03:49:51.447077 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.447052 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Nov 04 03:49:51.447211 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.447093 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Nov 04 03:49:51.447211 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.447136 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Nov 04 03:49:51.454232 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.454187 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Nov 04 03:49:51.454485 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.454467 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-web-config\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.454643 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.454629 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.454803 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.454791 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.454945 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.454932 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e191ac4e-0058-44af-b530-5e68a054d946-config-out\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.455054 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.455040 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.455621 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.455602 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.455848 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.455826 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.456572 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.456534 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.456713 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.456679 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-config\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.456787 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.456714 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.456787 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.456732 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.456787 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.456749 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.456787 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.456784 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.456975 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.456825 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.456975 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.456867 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.456975 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.456894 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.463557 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.457655 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Nov 04 03:49:51.463557 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.457805 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bsczw\"" Nov 04 03:49:51.463557 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.457897 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Nov 04 03:49:51.463557 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.457975 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Nov 04 03:49:51.463557 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.458036 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Nov 04 03:49:51.463557 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.458114 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Nov 04 03:49:51.463557 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.458179 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Nov 04 03:49:51.463557 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.458285 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Nov 04 03:49:51.463557 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.458423 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Nov 04 03:49:51.463557 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.458464 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-bjr97m2gdpk1p\"" Nov 04 03:49:51.463557 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.457051 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlw9q\" (UniqueName: \"kubernetes.io/projected/e191ac4e-0058-44af-b530-5e68a054d946-kube-api-access-zlw9q\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.463557 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.460097 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e191ac4e-0058-44af-b530-5e68a054d946-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.463557 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.460132 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.463557 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.460161 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.463557 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.460485 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 03:49:51.537085 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.537008 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" event={"ID":"8a6918a9-3ff2-46aa-bc91-affd1835403d","Type":"ContainerStarted","Data":"c56c6fdec811d672aae0df9a60d56bfb8a60bf4a752af7090df76e5cbc50993f"} Nov 04 03:49:51.537230 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.537092 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" event={"ID":"8a6918a9-3ff2-46aa-bc91-affd1835403d","Type":"ContainerStarted","Data":"9733927d880f352131a357a78fe170a9540b18830bf8de03919ab4549e685ae1"} Nov 04 03:49:51.537230 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.537133 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" event={"ID":"8a6918a9-3ff2-46aa-bc91-affd1835403d","Type":"ContainerStarted","Data":"f703760a91688bcb3d15340fb36bc01372c43d34cd932fcddbd02bc0c8335274"} Nov 04 03:49:51.537397 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.537370 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:51.561578 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.561517 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.561779 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.561643 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.561779 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.561693 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e191ac4e-0058-44af-b530-5e68a054d946-config-out\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.561918 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.561824 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.561918 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.561862 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.561918 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.561908 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.562070 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.562000 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.562170 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.562143 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-config\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.562271 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.562258 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.562360 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.562347 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.562412 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.562374 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.562412 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.562401 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.562521 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.562494 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.562685 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.562670 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.568257 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.562782 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.568257 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.562877 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlw9q\" (UniqueName: \"kubernetes.io/projected/e191ac4e-0058-44af-b530-5e68a054d946-kube-api-access-zlw9q\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.568257 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.563060 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e191ac4e-0058-44af-b530-5e68a054d946-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.568257 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.563120 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.568257 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.563149 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.568257 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.563182 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-web-config\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.568257 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.563287 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.568257 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.565963 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-config\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.568257 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.566542 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.568257 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.567214 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.575001 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.574614 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.575001 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.574911 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.575262 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.575235 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.575694 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.575670 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.576190 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.576169 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.578841 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.578815 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e191ac4e-0058-44af-b530-5e68a054d946-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.579468 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.579444 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.579604 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.579576 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.580930 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.580910 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.581090 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.581067 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.581157 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.581125 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.584541 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.584513 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-web-config\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.584651 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.584624 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e191ac4e-0058-44af-b530-5e68a054d946-config-out\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.585489 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.585468 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.586218 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.586191 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlw9q\" (UniqueName: \"kubernetes.io/projected/e191ac4e-0058-44af-b530-5e68a054d946-kube-api-access-zlw9q\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.587675 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.587656 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:51.775985 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:51.775698 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:49:52.077535 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:52.076159 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" podStartSLOduration=1.866817041 podStartE2EDuration="5.076139549s" totalImagesPullingTime="3.209322508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:49:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:51.576933817 +0000 UTC m=+63.946394490" watchObservedRunningTime="2026-11-04 03:49:52.076139549 +0000 UTC m=+64.445600226" Nov 04 03:49:52.077535 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:52.077428 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 03:49:52.081118 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:49:52.081065 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode191ac4e_0058_44af_b530_5e68a054d946.slice/crio-1694654274d75c17fa3418ca8df7950e1bf8a3fd72c1096b885492cd16ddf7a7 WatchSource:0}: Error finding container 1694654274d75c17fa3418ca8df7950e1bf8a3fd72c1096b885492cd16ddf7a7: Status 404 returned error can't find the container with id 1694654274d75c17fa3418ca8df7950e1bf8a3fd72c1096b885492cd16ddf7a7 Nov 04 03:49:52.541164 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:52.541125 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e191ac4e-0058-44af-b530-5e68a054d946","Type":"ContainerStarted","Data":"1694654274d75c17fa3418ca8df7950e1bf8a3fd72c1096b885492cd16ddf7a7"} Nov 04 03:49:52.542639 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:52.542605 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-f92gv" event={"ID":"27390c92-0480-4b55-ba3d-e20f7ada5b83","Type":"ContainerStarted","Data":"f2e601786eef4d4538ea220802d42a94bdebf3b79f2d1a54d24d592c66426023"} Nov 04 03:49:52.542829 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:52.542810 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-f92gv" Nov 04 03:49:52.547631 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:52.547601 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-f92gv" Nov 04 03:49:52.562897 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:52.562851 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-f92gv" podStartSLOduration=1.7634162020000002 podStartE2EDuration="3.562835219s" totalImagesPullingTime="1.799419017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:49:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:52.562586968 +0000 UTC m=+64.932047809" watchObservedRunningTime="2026-11-04 03:49:52.562835219 +0000 UTC m=+64.932295895" Nov 04 03:49:53.501627 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:53.501592 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vcxpf" Nov 04 03:49:53.549383 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:53.549352 2575 generic.go:356] "Generic (PLEG): container finished" podID="e191ac4e-0058-44af-b530-5e68a054d946" containerID="30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad" exitCode=0 Nov 04 03:49:53.549831 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:53.549443 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e191ac4e-0058-44af-b530-5e68a054d946","Type":"ContainerDied","Data":"30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad"} Nov 04 03:49:53.784106 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:53.784011 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs\") pod \"network-metrics-daemon-p422k\" (UID: \"56348938-ef44-41ab-8a71-acc1e4308262\") " pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:49:53.787283 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:53.787257 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 04 03:49:53.796749 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:53.796723 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs\") pod \"network-metrics-daemon-p422k\" (UID: \"56348938-ef44-41ab-8a71-acc1e4308262\") " pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:49:53.885288 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:53.885246 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret\") pod \"global-pull-secret-syncer-p84rx\" (UID: \"06a274ba-93f4-4dbc-ac39-4039fd254e7d\") " pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:49:53.888575 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:53.888551 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 04 03:49:53.897982 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:53.897935 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret\") pod \"global-pull-secret-syncer-p84rx\" (UID: \"06a274ba-93f4-4dbc-ac39-4039fd254e7d\") " pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:49:53.985666 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:53.985632 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lvzv\" (UniqueName: \"kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv\") pod \"network-check-target-967tl\" (UID: \"5cf7e388-1678-4afb-8877-30179162401b\") " pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:49:53.988584 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:53.988563 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 04 03:49:53.999377 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:53.999351 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 04 03:49:54.020158 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:54.020123 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lvzv\" (UniqueName: \"kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv\") pod \"network-check-target-967tl\" (UID: \"5cf7e388-1678-4afb-8877-30179162401b\") " pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:49:54.081330 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:54.081303 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-p2j7s\"" Nov 04 03:49:54.088662 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:54.088627 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:49:54.090567 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:54.090540 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fmfbg\"" Nov 04 03:49:54.094216 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:54.094181 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-49g8t\"" Nov 04 03:49:54.098419 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:54.098398 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:49:54.101331 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:54.101300 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:49:54.396390 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:54.396206 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p84rx"] Nov 04 03:49:54.397979 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:54.397951 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-967tl"] Nov 04 03:49:54.405385 ip-10-0-129-134 kubenswrapper[2575]: W1104 03:49:54.405361 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06a274ba_93f4_4dbc_ac39_4039fd254e7d.slice/crio-65a07e471632b7210133b9bc4f9764e3c56e676e8dded2c4ce2b3943e3fddc6b WatchSource:0}: Error finding container 65a07e471632b7210133b9bc4f9764e3c56e676e8dded2c4ce2b3943e3fddc6b: Status 404 returned error can't find the container with id 65a07e471632b7210133b9bc4f9764e3c56e676e8dded2c4ce2b3943e3fddc6b Nov 04 03:49:54.415634 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:54.415607 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p422k"] Nov 04 03:49:54.555177 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:54.555140 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p422k" event={"ID":"56348938-ef44-41ab-8a71-acc1e4308262","Type":"ContainerStarted","Data":"6590bc15222877a1cd54c05547b745b132dddd26397761af261ce8473d3d7295"} Nov 04 03:49:54.556775 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:54.556730 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p84rx" event={"ID":"06a274ba-93f4-4dbc-ac39-4039fd254e7d","Type":"ContainerStarted","Data":"65a07e471632b7210133b9bc4f9764e3c56e676e8dded2c4ce2b3943e3fddc6b"} Nov 04 03:49:54.558103 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:54.558068 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-967tl" event={"ID":"5cf7e388-1678-4afb-8877-30179162401b","Type":"ContainerStarted","Data":"51a75d1eecffb2b94dbaec40d2e7567f10f4f26e844f2476fec0bd7357a87e38"} Nov 04 03:49:56.579430 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:56.578500 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p422k" event={"ID":"56348938-ef44-41ab-8a71-acc1e4308262","Type":"ContainerStarted","Data":"61764b97cc1a541980563939fbca24f802e0893482f2a2c36ada918684cb0675"} Nov 04 03:49:56.579430 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:56.578571 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p422k" event={"ID":"56348938-ef44-41ab-8a71-acc1e4308262","Type":"ContainerStarted","Data":"8e2b0061b5d42aae018ea80e1c137743ea59b5ac7bf4a0a77d420f7024f40fde"} Nov 04 03:49:56.600815 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:56.600329 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-p422k" podStartSLOduration=66.094553199 podStartE2EDuration="1m7.600311737s" totalImagesPullingTime="1.505758538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:48:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:56.60020106 +0000 UTC m=+68.969661738" watchObservedRunningTime="2026-11-04 03:49:56.600311737 +0000 UTC m=+68.969772409" Nov 04 03:49:57.550966 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:57.550936 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:49:58.604618 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:58.604591 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e191ac4e-0058-44af-b530-5e68a054d946","Type":"ContainerStarted","Data":"0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907"} Nov 04 03:49:58.604988 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:58.604655 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e191ac4e-0058-44af-b530-5e68a054d946","Type":"ContainerStarted","Data":"77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d"} Nov 04 03:49:58.604988 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:58.604681 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e191ac4e-0058-44af-b530-5e68a054d946","Type":"ContainerStarted","Data":"ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3"} Nov 04 03:49:59.613713 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:59.613671 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e191ac4e-0058-44af-b530-5e68a054d946","Type":"ContainerStarted","Data":"1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978"} Nov 04 03:49:59.614194 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:59.613786 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e191ac4e-0058-44af-b530-5e68a054d946","Type":"ContainerStarted","Data":"c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619"} Nov 04 03:49:59.614194 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:59.613871 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e191ac4e-0058-44af-b530-5e68a054d946","Type":"ContainerStarted","Data":"a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae"} Nov 04 03:49:59.653336 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:49:59.652578 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.589988585 podStartE2EDuration="8.652560464s" totalImagesPullingTime="6.046902557s" totalInitContainerRuntime="15.669322ms" isStatefulPod=false podCreationTimestamp="2026-11-04 03:49:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:59.651111905 +0000 UTC m=+72.020572582" watchObservedRunningTime="2026-11-04 03:49:59.652560464 +0000 UTC m=+72.022021142" Nov 04 03:50:00.620026 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:50:00.619923 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-967tl" event={"ID":"5cf7e388-1678-4afb-8877-30179162401b","Type":"ContainerStarted","Data":"f541321e615814f8db832e5878fe8f285290814b12ed0130ff63eaf10fed32a6"} Nov 04 03:50:00.620651 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:50:00.620625 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:50:00.639225 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:50:00.639132 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-967tl" podStartSLOduration=65.728404964 podStartE2EDuration="1m11.639118199s" totalImagesPullingTime="5.910713235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:48:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:50:00.638016826 +0000 UTC m=+73.007477559" watchObservedRunningTime="2026-11-04 03:50:00.639118199 +0000 UTC m=+73.008578912" Nov 04 03:50:01.775896 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:50:01.775850 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:01.898693 ip-10-0-129-134 kubenswrapper[2575]: I1104 03:50:01.898552 2575 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 03:50:01.898688 ip-10-0-129-134 systemd[1]: Stopping Kubernetes Kubelet... Nov 04 03:50:01.914209 ip-10-0-129-134 systemd[1]: kubelet.service: Deactivated successfully. Nov 04 03:50:01.914382 ip-10-0-129-134 systemd[1]: Stopped Kubernetes Kubelet. Nov 04 03:50:01.914435 ip-10-0-129-134 systemd[1]: kubelet.service: Consumed 4.159s CPU time. Nov 04 03:50:01.929566 ip-10-0-129-134 systemd[1]: Starting Kubernetes Kubelet... Nov 04 03:50:01.990881 ip-10-0-129-134 kubenswrapper[6000]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 03:50:01.990881 ip-10-0-129-134 kubenswrapper[6000]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Nov 04 03:50:01.990881 ip-10-0-129-134 kubenswrapper[6000]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 03:50:01.990881 ip-10-0-129-134 kubenswrapper[6000]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 03:50:01.993116 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993020 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 03:50:01.993116 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993112 6000 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 03:50:01.993116 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993116 6000 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 03:50:01.993116 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993120 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 03:50:01.993116 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993123 6000 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 03:50:01.993304 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993127 6000 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 03:50:01.993304 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993129 6000 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 03:50:01.993304 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993132 6000 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 03:50:01.993304 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993136 6000 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 03:50:01.993304 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993139 6000 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 03:50:01.993304 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993141 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 03:50:01.993304 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993145 6000 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 03:50:01.993304 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993147 6000 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 03:50:01.993304 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993150 6000 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 03:50:01.993304 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993153 6000 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 03:50:01.993304 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993156 6000 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 03:50:01.993304 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993158 6000 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 03:50:01.993304 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993161 6000 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 03:50:01.993304 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993164 6000 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 03:50:01.993304 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993166 6000 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 03:50:01.993304 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993169 6000 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 03:50:01.993304 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993171 6000 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 03:50:01.993304 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993174 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 03:50:01.993304 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993177 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 03:50:01.993856 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993180 6000 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 03:50:01.993856 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993182 6000 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 03:50:01.993856 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993192 6000 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 03:50:01.993856 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993196 6000 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 03:50:01.993856 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993199 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 03:50:01.993856 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993202 6000 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 03:50:01.993856 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993204 6000 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 03:50:01.993856 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993207 6000 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 03:50:01.993856 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993211 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 03:50:01.993856 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993214 6000 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 03:50:01.993856 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993216 6000 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 03:50:01.993856 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993219 6000 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 03:50:01.993856 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993222 6000 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 03:50:01.993856 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993226 6000 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 03:50:01.993856 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993231 6000 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 03:50:01.993856 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993234 6000 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 03:50:01.993856 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993237 6000 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 03:50:01.993856 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993239 6000 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 03:50:01.993856 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993242 6000 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 03:50:01.994355 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993245 6000 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 03:50:01.994355 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993247 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 03:50:01.994355 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993250 6000 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 03:50:01.994355 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993253 6000 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 03:50:01.994355 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993256 6000 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 03:50:01.994355 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993258 6000 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 03:50:01.994355 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993261 6000 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 03:50:01.994355 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993264 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 03:50:01.994355 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993266 6000 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 03:50:01.994355 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993269 6000 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 03:50:01.994355 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993272 6000 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 03:50:01.994355 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993275 6000 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 03:50:01.994355 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993277 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 03:50:01.994355 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993280 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 03:50:01.994355 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993283 6000 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 03:50:01.994355 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993285 6000 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 03:50:01.994355 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993288 6000 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 03:50:01.994355 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993291 6000 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 03:50:01.994355 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993294 6000 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 03:50:01.994355 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993296 6000 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 03:50:01.994903 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993299 6000 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 03:50:01.994903 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993301 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 03:50:01.994903 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993304 6000 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 03:50:01.994903 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993307 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 03:50:01.994903 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993309 6000 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 03:50:01.994903 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993312 6000 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 03:50:01.994903 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993315 6000 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 03:50:01.994903 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993318 6000 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 03:50:01.994903 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993320 6000 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 03:50:01.994903 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993323 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 03:50:01.994903 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993326 6000 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 03:50:01.994903 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993328 6000 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 03:50:01.994903 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993331 6000 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 03:50:01.994903 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993334 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 03:50:01.994903 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993337 6000 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 03:50:01.994903 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993339 6000 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 03:50:01.994903 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993342 6000 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 03:50:01.994903 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993345 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 03:50:01.994903 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993348 6000 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 03:50:01.994903 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993351 6000 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 03:50:01.995426 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993353 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 03:50:01.995426 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993356 6000 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 03:50:01.995426 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993358 6000 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 03:50:01.995426 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993361 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 03:50:01.995426 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993364 6000 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 03:50:01.995426 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993366 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 03:50:01.995426 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993369 6000 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 03:50:01.995426 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993372 6000 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 03:50:01.995426 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993375 6000 feature_gate.go:386] unrecognized feature gate: Example Nov 04 03:50:01.995426 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993378 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 03:50:01.995426 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993380 6000 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 03:50:01.995426 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993383 6000 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 03:50:01.995426 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993385 6000 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 03:50:01.995426 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993388 6000 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 03:50:01.995426 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993391 6000 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 03:50:01.995426 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993394 6000 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 03:50:01.995426 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993397 6000 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 03:50:01.995426 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993402 6000 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 03:50:01.995426 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993405 6000 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 03:50:01.995426 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993409 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 03:50:01.995928 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993412 6000 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 03:50:01.995928 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993415 6000 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 03:50:01.995928 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.993418 6000 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 03:50:01.995928 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994232 6000 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 03:50:01.995928 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994241 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 03:50:01.995928 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994244 6000 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 03:50:01.995928 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994248 6000 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 03:50:01.995928 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994251 6000 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 03:50:01.995928 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994255 6000 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 03:50:01.995928 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994257 6000 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 03:50:01.995928 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994260 6000 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 03:50:01.995928 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994263 6000 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 03:50:01.995928 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994266 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 03:50:01.995928 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994269 6000 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 03:50:01.995928 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994273 6000 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 03:50:01.995928 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994276 6000 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 03:50:01.995928 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994279 6000 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 03:50:01.995928 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994282 6000 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 03:50:01.995928 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994285 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 03:50:01.996430 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994288 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 03:50:01.996430 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994291 6000 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 03:50:01.996430 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994294 6000 feature_gate.go:386] unrecognized feature gate: Example Nov 04 03:50:01.996430 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994297 6000 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 03:50:01.996430 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994299 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 03:50:01.996430 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994302 6000 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 03:50:01.996430 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994305 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 03:50:01.996430 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994307 6000 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 03:50:01.996430 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994310 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 03:50:01.996430 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994313 6000 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 03:50:01.996430 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994315 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 03:50:01.996430 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994318 6000 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 03:50:01.996430 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994321 6000 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 03:50:01.996430 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994323 6000 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 03:50:01.996430 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994326 6000 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 03:50:01.996430 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994329 6000 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 03:50:01.996430 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994332 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 03:50:01.996430 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994335 6000 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 03:50:01.996430 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994338 6000 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 03:50:01.996430 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994341 6000 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 03:50:01.996948 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994344 6000 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 03:50:01.996948 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994346 6000 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 03:50:01.996948 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994349 6000 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 03:50:01.996948 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994351 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 03:50:01.996948 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994354 6000 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 03:50:01.996948 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994357 6000 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 03:50:01.996948 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994359 6000 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 03:50:01.996948 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994362 6000 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 03:50:01.996948 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994365 6000 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 03:50:01.996948 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994367 6000 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 03:50:01.996948 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994370 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 03:50:01.996948 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994373 6000 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 03:50:01.996948 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994376 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 03:50:01.996948 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994380 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 03:50:01.996948 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994382 6000 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 03:50:01.996948 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994385 6000 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 03:50:01.996948 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994388 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 03:50:01.996948 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994391 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 03:50:01.996948 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994394 6000 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 03:50:01.997407 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994396 6000 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 03:50:01.997407 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994399 6000 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 03:50:01.997407 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994402 6000 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 03:50:01.997407 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994404 6000 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 03:50:01.997407 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994409 6000 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 03:50:01.997407 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994413 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 03:50:01.997407 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994417 6000 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 03:50:01.997407 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994421 6000 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 03:50:01.997407 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994425 6000 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 03:50:01.997407 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994430 6000 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 03:50:01.997407 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994433 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 03:50:01.997407 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994436 6000 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 03:50:01.997407 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994438 6000 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 03:50:01.997407 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994441 6000 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 03:50:01.997407 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994444 6000 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 03:50:01.997407 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994448 6000 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 03:50:01.997407 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994451 6000 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 03:50:01.997407 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994454 6000 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 03:50:01.997407 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994456 6000 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 03:50:01.997407 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994459 6000 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 03:50:01.997953 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994462 6000 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 03:50:01.997953 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994465 6000 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 03:50:01.997953 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994467 6000 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 03:50:01.997953 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994470 6000 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 03:50:01.997953 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994474 6000 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 03:50:01.997953 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994478 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 03:50:01.997953 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994482 6000 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 03:50:01.997953 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994485 6000 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 03:50:01.997953 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994490 6000 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 03:50:01.997953 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994494 6000 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 03:50:01.997953 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994499 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 03:50:01.997953 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994503 6000 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 03:50:01.997953 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994506 6000 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 03:50:01.997953 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994509 6000 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 03:50:01.997953 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994512 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 03:50:01.997953 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994515 6000 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 03:50:01.997953 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994517 6000 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 03:50:01.997953 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994520 6000 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 03:50:01.997953 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994523 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 03:50:01.998435 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994526 6000 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 03:50:01.998435 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994529 6000 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 03:50:01.998435 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994531 6000 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 03:50:01.998435 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994535 6000 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 03:50:01.998435 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994538 6000 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 03:50:01.998435 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994540 6000 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 03:50:01.998435 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994543 6000 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 03:50:01.998435 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994547 6000 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 03:50:01.998435 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994550 6000 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 03:50:01.998435 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994552 6000 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 03:50:01.998435 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994555 6000 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 03:50:01.998435 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.994558 6000 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 03:50:01.998435 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995090 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 03:50:01.998435 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995096 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 03:50:01.998435 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995099 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 03:50:01.998435 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995102 6000 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 03:50:01.998435 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995104 6000 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 03:50:01.998435 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995107 6000 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 03:50:01.998435 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995110 6000 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 03:50:01.998435 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995113 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 03:50:01.998950 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995116 6000 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 03:50:01.998950 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995119 6000 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 03:50:01.998950 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995122 6000 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 03:50:01.998950 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995125 6000 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 03:50:01.998950 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995128 6000 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 03:50:01.998950 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995130 6000 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 03:50:01.998950 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995133 6000 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 03:50:01.998950 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995136 6000 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 03:50:01.998950 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995138 6000 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 03:50:01.998950 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995141 6000 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 03:50:01.998950 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995143 6000 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 03:50:01.998950 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995146 6000 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 03:50:01.998950 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995150 6000 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 03:50:01.998950 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995152 6000 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 03:50:01.998950 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995155 6000 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 03:50:01.998950 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995159 6000 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 03:50:01.998950 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995162 6000 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 03:50:01.998950 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995164 6000 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 03:50:01.998950 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995167 6000 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 03:50:01.998950 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995170 6000 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 03:50:01.999493 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995172 6000 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 03:50:01.999493 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995175 6000 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 03:50:01.999493 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995177 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 03:50:01.999493 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995180 6000 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 03:50:01.999493 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995183 6000 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 03:50:01.999493 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995185 6000 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 03:50:01.999493 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995188 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 03:50:01.999493 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995190 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 03:50:01.999493 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995193 6000 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 03:50:01.999493 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995196 6000 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 03:50:01.999493 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995199 6000 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 03:50:01.999493 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995201 6000 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 03:50:01.999493 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995204 6000 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 03:50:01.999493 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995207 6000 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 03:50:01.999493 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995209 6000 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 03:50:01.999493 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995212 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 03:50:01.999493 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995215 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 03:50:01.999493 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995217 6000 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 03:50:01.999493 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995220 6000 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 03:50:01.999493 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995223 6000 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 03:50:02.000289 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995226 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 03:50:02.000289 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995228 6000 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 03:50:02.000289 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995231 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 03:50:02.000289 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995234 6000 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 03:50:02.000289 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995237 6000 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 03:50:02.000289 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995239 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 03:50:02.000289 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995242 6000 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 03:50:02.000289 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995244 6000 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 03:50:02.000289 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995247 6000 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 03:50:02.000289 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995250 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 03:50:02.000289 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995253 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 03:50:02.000289 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995255 6000 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 03:50:02.000289 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995258 6000 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 03:50:02.000289 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995260 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 03:50:02.000289 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995263 6000 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 03:50:02.000289 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995265 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 03:50:02.000289 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995268 6000 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 03:50:02.000289 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995270 6000 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 03:50:02.000289 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995273 6000 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 03:50:02.000289 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995276 6000 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 03:50:02.001066 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995278 6000 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 03:50:02.001066 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995281 6000 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 03:50:02.001066 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995284 6000 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 03:50:02.001066 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995286 6000 feature_gate.go:386] unrecognized feature gate: Example Nov 04 03:50:02.001066 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995290 6000 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 03:50:02.001066 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995294 6000 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 03:50:02.001066 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995298 6000 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 03:50:02.001066 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995301 6000 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 03:50:02.001066 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995317 6000 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 03:50:02.001066 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995321 6000 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 03:50:02.001066 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995324 6000 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 03:50:02.001066 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995327 6000 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 03:50:02.001066 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995330 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 03:50:02.001066 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995333 6000 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 03:50:02.001066 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995335 6000 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 03:50:02.001066 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995338 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 03:50:02.001066 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995341 6000 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 03:50:02.001066 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995343 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 03:50:02.001066 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995346 6000 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 03:50:02.001893 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995350 6000 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 03:50:02.001893 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995353 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 03:50:02.001893 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995361 6000 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 03:50:02.001893 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995364 6000 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 03:50:02.001893 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995366 6000 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 03:50:02.001893 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995369 6000 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 03:50:02.001893 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995372 6000 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 03:50:02.001893 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995375 6000 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 03:50:02.001893 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995377 6000 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 03:50:02.001893 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995380 6000 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 03:50:02.001893 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995382 6000 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 03:50:02.001893 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995385 6000 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 03:50:02.001893 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995387 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 03:50:02.001893 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995390 6000 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 03:50:02.001893 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995393 6000 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 03:50:02.001893 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995395 6000 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 03:50:02.001893 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995398 6000 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 03:50:02.001893 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995401 6000 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 03:50:02.001893 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.995403 6000 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 03:50:02.002821 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995494 6000 flags.go:64] FLAG: --address="0.0.0.0" Nov 04 03:50:02.002821 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995501 6000 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Nov 04 03:50:02.002821 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995508 6000 flags.go:64] FLAG: --anonymous-auth="true" Nov 04 03:50:02.002821 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995515 6000 flags.go:64] FLAG: --application-metrics-count-limit="100" Nov 04 03:50:02.002821 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995522 6000 flags.go:64] FLAG: --authentication-token-webhook="false" Nov 04 03:50:02.002821 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995527 6000 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Nov 04 03:50:02.002821 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995532 6000 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Nov 04 03:50:02.002821 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995537 6000 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Nov 04 03:50:02.002821 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995540 6000 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Nov 04 03:50:02.002821 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995544 6000 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Nov 04 03:50:02.002821 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995547 6000 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Nov 04 03:50:02.002821 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995550 6000 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Nov 04 03:50:02.002821 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995554 6000 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Nov 04 03:50:02.002821 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995557 6000 flags.go:64] FLAG: --cgroup-root="" Nov 04 03:50:02.002821 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995560 6000 flags.go:64] FLAG: --cgroups-per-qos="true" Nov 04 03:50:02.002821 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995563 6000 flags.go:64] FLAG: --client-ca-file="" Nov 04 03:50:02.002821 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995567 6000 flags.go:64] FLAG: --cloud-provider="external" Nov 04 03:50:02.002821 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995574 6000 flags.go:64] FLAG: --cluster-dns="[]" Nov 04 03:50:02.002821 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995578 6000 flags.go:64] FLAG: --cluster-domain="" Nov 04 03:50:02.002821 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995581 6000 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Nov 04 03:50:02.002821 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995584 6000 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Nov 04 03:50:02.002821 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995588 6000 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Nov 04 03:50:02.002821 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995591 6000 flags.go:64] FLAG: --container-log-max-files="5" Nov 04 03:50:02.002821 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995595 6000 flags.go:64] FLAG: --container-log-max-size="10Mi" Nov 04 03:50:02.003896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995599 6000 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Nov 04 03:50:02.003896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995602 6000 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Nov 04 03:50:02.003896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995605 6000 flags.go:64] FLAG: --containerd-namespace="k8s.io" Nov 04 03:50:02.003896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995609 6000 flags.go:64] FLAG: --contention-profiling="false" Nov 04 03:50:02.003896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995612 6000 flags.go:64] FLAG: --cpu-cfs-quota="true" Nov 04 03:50:02.003896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995615 6000 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Nov 04 03:50:02.003896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995619 6000 flags.go:64] FLAG: --cpu-manager-policy="none" Nov 04 03:50:02.003896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995622 6000 flags.go:64] FLAG: --cpu-manager-policy-options="" Nov 04 03:50:02.003896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995627 6000 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Nov 04 03:50:02.003896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995630 6000 flags.go:64] FLAG: --enable-controller-attach-detach="true" Nov 04 03:50:02.003896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995633 6000 flags.go:64] FLAG: --enable-debugging-handlers="true" Nov 04 03:50:02.003896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995636 6000 flags.go:64] FLAG: --enable-load-reader="false" Nov 04 03:50:02.003896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995640 6000 flags.go:64] FLAG: --enable-server="true" Nov 04 03:50:02.003896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995643 6000 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Nov 04 03:50:02.003896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995648 6000 flags.go:64] FLAG: --event-burst="100" Nov 04 03:50:02.003896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995651 6000 flags.go:64] FLAG: --event-qps="50" Nov 04 03:50:02.003896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995654 6000 flags.go:64] FLAG: --event-storage-age-limit="default=0" Nov 04 03:50:02.003896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995658 6000 flags.go:64] FLAG: --event-storage-event-limit="default=0" Nov 04 03:50:02.003896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995661 6000 flags.go:64] FLAG: --eviction-hard="" Nov 04 03:50:02.003896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995665 6000 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Nov 04 03:50:02.003896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995668 6000 flags.go:64] FLAG: --eviction-minimum-reclaim="" Nov 04 03:50:02.003896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995671 6000 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Nov 04 03:50:02.003896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995674 6000 flags.go:64] FLAG: --eviction-soft="" Nov 04 03:50:02.003896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995677 6000 flags.go:64] FLAG: --eviction-soft-grace-period="" Nov 04 03:50:02.003896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995680 6000 flags.go:64] FLAG: --exit-on-lock-contention="false" Nov 04 03:50:02.004979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995685 6000 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Nov 04 03:50:02.004979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995689 6000 flags.go:64] FLAG: --experimental-mounter-path="" Nov 04 03:50:02.004979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995692 6000 flags.go:64] FLAG: --fail-cgroupv1="true" Nov 04 03:50:02.004979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995695 6000 flags.go:64] FLAG: --fail-swap-on="true" Nov 04 03:50:02.004979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995698 6000 flags.go:64] FLAG: --feature-gates="" Nov 04 03:50:02.004979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995702 6000 flags.go:64] FLAG: --file-check-frequency="20s" Nov 04 03:50:02.004979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995705 6000 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Nov 04 03:50:02.004979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995709 6000 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Nov 04 03:50:02.004979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995712 6000 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Nov 04 03:50:02.004979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995715 6000 flags.go:64] FLAG: --healthz-port="10248" Nov 04 03:50:02.004979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995718 6000 flags.go:64] FLAG: --help="false" Nov 04 03:50:02.004979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995721 6000 flags.go:64] FLAG: --hostname-override="ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:50:02.004979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995724 6000 flags.go:64] FLAG: --housekeeping-interval="10s" Nov 04 03:50:02.004979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995729 6000 flags.go:64] FLAG: --http-check-frequency="20s" Nov 04 03:50:02.004979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995734 6000 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Nov 04 03:50:02.004979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995739 6000 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Nov 04 03:50:02.004979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995743 6000 flags.go:64] FLAG: --image-gc-high-threshold="85" Nov 04 03:50:02.004979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995746 6000 flags.go:64] FLAG: --image-gc-low-threshold="80" Nov 04 03:50:02.004979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995749 6000 flags.go:64] FLAG: --image-service-endpoint="" Nov 04 03:50:02.004979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995767 6000 flags.go:64] FLAG: --kernel-memcg-notification="false" Nov 04 03:50:02.004979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995771 6000 flags.go:64] FLAG: --kube-api-burst="100" Nov 04 03:50:02.004979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995774 6000 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Nov 04 03:50:02.004979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995777 6000 flags.go:64] FLAG: --kube-api-qps="50" Nov 04 03:50:02.006038 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995780 6000 flags.go:64] FLAG: --kube-reserved="" Nov 04 03:50:02.006038 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995783 6000 flags.go:64] FLAG: --kube-reserved-cgroup="" Nov 04 03:50:02.006038 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995786 6000 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Nov 04 03:50:02.006038 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995789 6000 flags.go:64] FLAG: --kubelet-cgroups="" Nov 04 03:50:02.006038 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995792 6000 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Nov 04 03:50:02.006038 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995799 6000 flags.go:64] FLAG: --lock-file="" Nov 04 03:50:02.006038 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995802 6000 flags.go:64] FLAG: --log-cadvisor-usage="false" Nov 04 03:50:02.006038 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995805 6000 flags.go:64] FLAG: --log-flush-frequency="5s" Nov 04 03:50:02.006038 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995808 6000 flags.go:64] FLAG: --log-json-info-buffer-size="0" Nov 04 03:50:02.006038 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995816 6000 flags.go:64] FLAG: --log-json-split-stream="false" Nov 04 03:50:02.006038 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995820 6000 flags.go:64] FLAG: --log-text-info-buffer-size="0" Nov 04 03:50:02.006038 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995823 6000 flags.go:64] FLAG: --log-text-split-stream="false" Nov 04 03:50:02.006038 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995826 6000 flags.go:64] FLAG: --logging-format="text" Nov 04 03:50:02.006038 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995830 6000 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Nov 04 03:50:02.006038 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995833 6000 flags.go:64] FLAG: --make-iptables-util-chains="true" Nov 04 03:50:02.006038 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995836 6000 flags.go:64] FLAG: --manifest-url="" Nov 04 03:50:02.006038 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995839 6000 flags.go:64] FLAG: --manifest-url-header="" Nov 04 03:50:02.006038 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995844 6000 flags.go:64] FLAG: --max-housekeeping-interval="15s" Nov 04 03:50:02.006038 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995847 6000 flags.go:64] FLAG: --max-open-files="1000000" Nov 04 03:50:02.006038 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995851 6000 flags.go:64] FLAG: --max-pods="110" Nov 04 03:50:02.006038 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995854 6000 flags.go:64] FLAG: --maximum-dead-containers="-1" Nov 04 03:50:02.006038 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995857 6000 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Nov 04 03:50:02.006038 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995860 6000 flags.go:64] FLAG: --memory-manager-policy="None" Nov 04 03:50:02.006038 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995863 6000 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Nov 04 03:50:02.006038 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995866 6000 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995870 6000 flags.go:64] FLAG: --node-ip="0.0.0.0" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995873 6000 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995880 6000 flags.go:64] FLAG: --node-status-max-images="50" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995883 6000 flags.go:64] FLAG: --node-status-update-frequency="10s" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995886 6000 flags.go:64] FLAG: --oom-score-adj="-999" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995889 6000 flags.go:64] FLAG: --pod-cidr="" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995892 6000 flags.go:64] FLAG: --pod-manifest-path="" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995895 6000 flags.go:64] FLAG: --pod-max-pids="-1" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995898 6000 flags.go:64] FLAG: --pods-per-core="0" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995901 6000 flags.go:64] FLAG: --port="10250" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995905 6000 flags.go:64] FLAG: --protect-kernel-defaults="false" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995908 6000 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-09942282de6148589" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995912 6000 flags.go:64] FLAG: --qos-reserved="" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995915 6000 flags.go:64] FLAG: --read-only-port="10255" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995919 6000 flags.go:64] FLAG: --register-node="true" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995922 6000 flags.go:64] FLAG: --register-with-taints="" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995928 6000 flags.go:64] FLAG: --registry-burst="10" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995932 6000 flags.go:64] FLAG: --registry-qps="5" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995936 6000 flags.go:64] FLAG: --reserved-cpus="" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995939 6000 flags.go:64] FLAG: --reserved-memory="" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995944 6000 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995947 6000 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995951 6000 flags.go:64] FLAG: --rotate-certificates="false" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995954 6000 flags.go:64] FLAG: --rotate-server-certificates="false" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995957 6000 flags.go:64] FLAG: --runonce="false" Nov 04 03:50:02.007125 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995960 6000 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Nov 04 03:50:02.008336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995963 6000 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Nov 04 03:50:02.008336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995967 6000 flags.go:64] FLAG: --seccomp-default="false" Nov 04 03:50:02.008336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995970 6000 flags.go:64] FLAG: --serialize-image-pulls="true" Nov 04 03:50:02.008336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995973 6000 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Nov 04 03:50:02.008336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995976 6000 flags.go:64] FLAG: --storage-driver-db="cadvisor" Nov 04 03:50:02.008336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995979 6000 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Nov 04 03:50:02.008336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995982 6000 flags.go:64] FLAG: --storage-driver-password="root" Nov 04 03:50:02.008336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995985 6000 flags.go:64] FLAG: --storage-driver-secure="false" Nov 04 03:50:02.008336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995988 6000 flags.go:64] FLAG: --storage-driver-table="stats" Nov 04 03:50:02.008336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995991 6000 flags.go:64] FLAG: --storage-driver-user="root" Nov 04 03:50:02.008336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995994 6000 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Nov 04 03:50:02.008336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.995997 6000 flags.go:64] FLAG: --sync-frequency="1m0s" Nov 04 03:50:02.008336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.996000 6000 flags.go:64] FLAG: --system-cgroups="" Nov 04 03:50:02.008336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.996003 6000 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Nov 04 03:50:02.008336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.996008 6000 flags.go:64] FLAG: --system-reserved-cgroup="" Nov 04 03:50:02.008336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.996011 6000 flags.go:64] FLAG: --tls-cert-file="" Nov 04 03:50:02.008336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.996014 6000 flags.go:64] FLAG: --tls-cipher-suites="[]" Nov 04 03:50:02.008336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.996019 6000 flags.go:64] FLAG: --tls-min-version="" Nov 04 03:50:02.008336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.996023 6000 flags.go:64] FLAG: --tls-private-key-file="" Nov 04 03:50:02.008336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.996026 6000 flags.go:64] FLAG: --topology-manager-policy="none" Nov 04 03:50:02.008336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.996029 6000 flags.go:64] FLAG: --topology-manager-policy-options="" Nov 04 03:50:02.008336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.996032 6000 flags.go:64] FLAG: --topology-manager-scope="container" Nov 04 03:50:02.008336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.996037 6000 flags.go:64] FLAG: --v="2" Nov 04 03:50:02.008336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.996041 6000 flags.go:64] FLAG: --version="false" Nov 04 03:50:02.008336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.996045 6000 flags.go:64] FLAG: --vmodule="" Nov 04 03:50:02.009391 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.996053 6000 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Nov 04 03:50:02.009391 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.996057 6000 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Nov 04 03:50:02.009391 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996192 6000 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 03:50:02.009391 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996196 6000 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 03:50:02.009391 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996199 6000 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 03:50:02.009391 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996202 6000 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 03:50:02.009391 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996205 6000 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 03:50:02.009391 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996208 6000 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 03:50:02.009391 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996211 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 03:50:02.009391 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996214 6000 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 03:50:02.009391 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996217 6000 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 03:50:02.009391 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996219 6000 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 03:50:02.009391 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996222 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 03:50:02.009391 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996225 6000 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 03:50:02.009391 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996228 6000 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 03:50:02.009391 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996230 6000 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 03:50:02.009391 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996233 6000 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 03:50:02.009391 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996235 6000 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 03:50:02.009391 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996238 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 03:50:02.010031 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996241 6000 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 03:50:02.010031 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996243 6000 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 03:50:02.010031 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996246 6000 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 03:50:02.010031 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996248 6000 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 03:50:02.010031 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996251 6000 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 03:50:02.010031 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996254 6000 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 03:50:02.010031 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996259 6000 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 03:50:02.010031 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996262 6000 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 03:50:02.010031 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996265 6000 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 03:50:02.010031 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996267 6000 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 03:50:02.010031 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996271 6000 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 03:50:02.010031 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996274 6000 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 03:50:02.010031 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996277 6000 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 03:50:02.010031 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996279 6000 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 03:50:02.010031 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996283 6000 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 03:50:02.010031 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996285 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 03:50:02.010031 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996288 6000 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 03:50:02.010031 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996291 6000 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 03:50:02.010031 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996293 6000 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 03:50:02.010031 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996296 6000 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 03:50:02.010593 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996299 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 03:50:02.010593 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996301 6000 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 03:50:02.010593 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996304 6000 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 03:50:02.010593 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996306 6000 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 03:50:02.010593 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996309 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 03:50:02.010593 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996311 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 03:50:02.010593 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996315 6000 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 03:50:02.010593 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996320 6000 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 03:50:02.010593 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996324 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 03:50:02.010593 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996327 6000 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 03:50:02.010593 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996330 6000 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 03:50:02.010593 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996333 6000 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 03:50:02.010593 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996335 6000 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 03:50:02.010593 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996338 6000 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 03:50:02.010593 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996340 6000 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 03:50:02.010593 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996343 6000 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 03:50:02.010593 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996345 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 03:50:02.010593 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996348 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 03:50:02.011117 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996352 6000 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 03:50:02.011117 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996354 6000 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 03:50:02.011117 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996357 6000 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 03:50:02.011117 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996359 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 03:50:02.011117 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996363 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 03:50:02.011117 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996366 6000 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 03:50:02.011117 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996369 6000 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 03:50:02.011117 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996371 6000 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 03:50:02.011117 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996374 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 03:50:02.011117 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996377 6000 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 03:50:02.011117 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996380 6000 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 03:50:02.011117 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996382 6000 feature_gate.go:386] unrecognized feature gate: Example Nov 04 03:50:02.011117 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996385 6000 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 03:50:02.011117 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996388 6000 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 03:50:02.011117 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996390 6000 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 03:50:02.011117 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996393 6000 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 03:50:02.011117 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996395 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 03:50:02.011117 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996398 6000 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 03:50:02.011117 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996400 6000 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 03:50:02.011117 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996403 6000 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 03:50:02.011613 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996406 6000 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 03:50:02.011613 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996408 6000 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 03:50:02.011613 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996411 6000 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 03:50:02.011613 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996413 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 03:50:02.011613 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996416 6000 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 03:50:02.011613 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996418 6000 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 03:50:02.011613 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996421 6000 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 03:50:02.011613 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996423 6000 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 03:50:02.011613 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996426 6000 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 03:50:02.011613 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996429 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 03:50:02.011613 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996431 6000 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 03:50:02.011613 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996434 6000 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 03:50:02.011613 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996438 6000 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 03:50:02.011613 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996441 6000 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 03:50:02.011613 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996444 6000 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 03:50:02.011613 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996446 6000 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 03:50:02.011613 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996450 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 03:50:02.011613 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996453 6000 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 03:50:02.011613 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996455 6000 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 03:50:02.011613 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996458 6000 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 03:50:02.012252 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996460 6000 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 03:50:02.012252 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996463 6000 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 03:50:02.012252 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996466 6000 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 03:50:02.012252 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996469 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 03:50:02.012252 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996471 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 03:50:02.012252 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996474 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 03:50:02.012252 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996476 6000 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 03:50:02.012252 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996479 6000 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 03:50:02.012252 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996482 6000 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 03:50:02.012252 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996484 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 03:50:02.012252 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996487 6000 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 03:50:02.012252 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:01.996513 6000 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 03:50:02.012252 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996547 6000 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 03:50:02.012252 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996551 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 03:50:02.012252 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996554 6000 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 03:50:02.012729 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996557 6000 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 03:50:02.012729 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996560 6000 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 03:50:02.012729 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996563 6000 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 03:50:02.012729 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996566 6000 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 03:50:02.012729 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996568 6000 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 03:50:02.012729 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996571 6000 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 03:50:02.012729 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996574 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 03:50:02.012729 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996577 6000 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 03:50:02.012729 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996579 6000 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 03:50:02.012729 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996582 6000 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 03:50:02.012729 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996586 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 03:50:02.012729 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996589 6000 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 03:50:02.012729 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996591 6000 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 03:50:02.012729 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996596 6000 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 03:50:02.012729 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996599 6000 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 03:50:02.012729 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996602 6000 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 03:50:02.012729 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996604 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 03:50:02.012729 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996607 6000 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 03:50:02.012729 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996609 6000 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 03:50:02.013350 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996612 6000 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 03:50:02.013350 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996615 6000 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 03:50:02.013350 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996618 6000 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 03:50:02.013350 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996620 6000 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 03:50:02.013350 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996623 6000 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 03:50:02.013350 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996626 6000 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 03:50:02.013350 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996628 6000 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 03:50:02.013350 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996631 6000 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 03:50:02.013350 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996633 6000 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 03:50:02.013350 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996636 6000 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 03:50:02.013350 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996638 6000 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 03:50:02.013350 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996641 6000 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 03:50:02.013350 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996644 6000 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 03:50:02.013350 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996646 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 03:50:02.013350 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996649 6000 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 03:50:02.013350 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996652 6000 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 03:50:02.013350 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996654 6000 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 03:50:02.013350 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996657 6000 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 03:50:02.013350 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996659 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 03:50:02.013350 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996662 6000 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 03:50:02.013863 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996664 6000 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 03:50:02.013863 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996667 6000 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 03:50:02.013863 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996669 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 03:50:02.013863 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996673 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 03:50:02.013863 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996677 6000 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 03:50:02.013863 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996682 6000 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 03:50:02.013863 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996688 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 03:50:02.013863 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996691 6000 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 03:50:02.013863 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996694 6000 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 03:50:02.013863 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996697 6000 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 03:50:02.013863 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996700 6000 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 03:50:02.013863 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996703 6000 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 03:50:02.013863 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996705 6000 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 03:50:02.013863 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996708 6000 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 03:50:02.013863 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996711 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 03:50:02.013863 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996714 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 03:50:02.013863 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996717 6000 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 03:50:02.013863 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996720 6000 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 03:50:02.014300 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996722 6000 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 03:50:02.014300 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996725 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 03:50:02.014300 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996727 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 03:50:02.014300 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996730 6000 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 03:50:02.014300 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996732 6000 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 03:50:02.014300 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996735 6000 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 03:50:02.014300 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996738 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 03:50:02.014300 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996740 6000 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 03:50:02.014300 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996743 6000 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 03:50:02.014300 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996745 6000 feature_gate.go:386] unrecognized feature gate: Example Nov 04 03:50:02.014300 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996748 6000 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 03:50:02.014300 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996750 6000 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 03:50:02.014300 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996767 6000 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 03:50:02.014300 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996770 6000 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 03:50:02.014300 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996773 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 03:50:02.014300 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996775 6000 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 03:50:02.014300 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996778 6000 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 03:50:02.014300 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996782 6000 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 03:50:02.014300 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996785 6000 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 03:50:02.014300 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996788 6000 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 03:50:02.014803 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996792 6000 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 03:50:02.014803 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996795 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 03:50:02.014803 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996797 6000 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 03:50:02.014803 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996800 6000 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 03:50:02.014803 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996803 6000 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 03:50:02.014803 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996805 6000 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 03:50:02.014803 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996808 6000 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 03:50:02.014803 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996811 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 03:50:02.014803 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996813 6000 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 03:50:02.014803 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996816 6000 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 03:50:02.014803 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996820 6000 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 03:50:02.014803 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996822 6000 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 03:50:02.014803 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996825 6000 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 03:50:02.014803 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996827 6000 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 03:50:02.014803 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996830 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 03:50:02.014803 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996833 6000 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 03:50:02.014803 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996835 6000 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 03:50:02.014803 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996838 6000 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 03:50:02.014803 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996840 6000 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 03:50:02.014803 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996844 6000 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 03:50:02.015290 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996847 6000 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 03:50:02.015290 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996850 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 03:50:02.015290 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996852 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 03:50:02.015290 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996854 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 03:50:02.015290 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996857 6000 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 03:50:02.015290 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:01.996861 6000 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 03:50:02.015290 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.002293 6000 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Nov 04 03:50:02.015290 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.002318 6000 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Nov 04 03:50:02.015290 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.002346 6000 watchdog_linux.go:95] "Systemd watchdog is not enabled" Nov 04 03:50:02.015290 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.002354 6000 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Nov 04 03:50:02.015290 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002449 6000 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 03:50:02.015290 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002457 6000 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 03:50:02.015290 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002463 6000 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 03:50:02.015290 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002468 6000 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 03:50:02.015290 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002473 6000 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 03:50:02.015290 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002480 6000 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 03:50:02.015290 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002486 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 03:50:02.015290 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002491 6000 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 03:50:02.015732 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002496 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 03:50:02.015732 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002500 6000 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 03:50:02.015732 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002505 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 03:50:02.015732 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002509 6000 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 03:50:02.015732 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002514 6000 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 03:50:02.015732 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002518 6000 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 03:50:02.015732 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002523 6000 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 03:50:02.015732 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002527 6000 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 03:50:02.015732 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002532 6000 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 03:50:02.015732 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002537 6000 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 03:50:02.015732 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002541 6000 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 03:50:02.015732 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002546 6000 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 03:50:02.015732 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002550 6000 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 03:50:02.015732 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002555 6000 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 03:50:02.015732 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002559 6000 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 03:50:02.015732 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002563 6000 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 03:50:02.015732 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002568 6000 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 03:50:02.015732 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002573 6000 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 03:50:02.015732 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002578 6000 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 03:50:02.015732 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002584 6000 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 03:50:02.016233 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002588 6000 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 03:50:02.016233 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002592 6000 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 03:50:02.016233 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002597 6000 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 03:50:02.016233 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002601 6000 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 03:50:02.016233 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002606 6000 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 03:50:02.016233 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002610 6000 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 03:50:02.016233 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002614 6000 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 03:50:02.016233 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002619 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 03:50:02.016233 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002623 6000 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 03:50:02.016233 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002628 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 03:50:02.016233 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002632 6000 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 03:50:02.016233 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002637 6000 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 03:50:02.016233 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002641 6000 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 03:50:02.016233 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002645 6000 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 03:50:02.016233 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002650 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 03:50:02.016233 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002654 6000 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 03:50:02.016233 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002659 6000 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 03:50:02.016233 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002664 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 03:50:02.016233 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002670 6000 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 03:50:02.016233 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002674 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 03:50:02.016713 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002678 6000 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 03:50:02.016713 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002682 6000 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 03:50:02.016713 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002687 6000 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 03:50:02.016713 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002691 6000 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 03:50:02.016713 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002695 6000 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 03:50:02.016713 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002700 6000 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 03:50:02.016713 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002704 6000 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 03:50:02.016713 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002708 6000 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 03:50:02.016713 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002715 6000 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 03:50:02.016713 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002722 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 03:50:02.016713 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002727 6000 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 03:50:02.016713 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002731 6000 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 03:50:02.016713 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002735 6000 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 03:50:02.016713 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002739 6000 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 03:50:02.016713 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002744 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 03:50:02.016713 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002748 6000 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 03:50:02.016713 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002776 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 03:50:02.016713 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002781 6000 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 03:50:02.016713 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002785 6000 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 03:50:02.017192 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002789 6000 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 03:50:02.017192 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002793 6000 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 03:50:02.017192 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002797 6000 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 03:50:02.017192 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002801 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 03:50:02.017192 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002806 6000 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 03:50:02.017192 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002810 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 03:50:02.017192 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002814 6000 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 03:50:02.017192 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002818 6000 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 03:50:02.017192 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002823 6000 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 03:50:02.017192 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002827 6000 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 03:50:02.017192 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002831 6000 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 03:50:02.017192 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002836 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 03:50:02.017192 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002841 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 03:50:02.017192 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002846 6000 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 03:50:02.017192 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002851 6000 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 03:50:02.017192 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002855 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 03:50:02.017192 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002859 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 03:50:02.017192 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002863 6000 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 03:50:02.017192 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002867 6000 feature_gate.go:386] unrecognized feature gate: Example Nov 04 03:50:02.017710 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002871 6000 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 03:50:02.017710 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002876 6000 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 03:50:02.017710 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002880 6000 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 03:50:02.017710 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002885 6000 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 03:50:02.017710 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002890 6000 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 03:50:02.017710 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002894 6000 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 03:50:02.017710 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002898 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 03:50:02.017710 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002903 6000 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 03:50:02.017710 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002907 6000 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 03:50:02.017710 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002911 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 03:50:02.017710 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002916 6000 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 03:50:02.017710 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002921 6000 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 03:50:02.017710 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002925 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 03:50:02.017710 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002929 6000 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 03:50:02.017710 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002934 6000 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 03:50:02.017710 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002938 6000 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 03:50:02.017710 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002942 6000 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 03:50:02.017710 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002947 6000 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 03:50:02.017710 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002951 6000 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 03:50:02.018434 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.002956 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 03:50:02.018434 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.003003 6000 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 03:50:02.018434 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003250 6000 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 03:50:02.018434 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003260 6000 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 03:50:02.018434 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003268 6000 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 03:50:02.018434 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003275 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 03:50:02.018434 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003281 6000 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 03:50:02.018434 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003288 6000 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 03:50:02.018434 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003293 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 03:50:02.018434 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003298 6000 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 03:50:02.018434 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003302 6000 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 03:50:02.018434 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003306 6000 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 03:50:02.018434 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003311 6000 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 03:50:02.018434 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003315 6000 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 03:50:02.018434 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003319 6000 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 03:50:02.019145 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003324 6000 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 03:50:02.019145 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003329 6000 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 03:50:02.019145 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003333 6000 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 03:50:02.019145 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003337 6000 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 03:50:02.019145 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003342 6000 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 03:50:02.019145 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003347 6000 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 03:50:02.019145 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003351 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 03:50:02.019145 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003355 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 03:50:02.019145 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003360 6000 feature_gate.go:386] unrecognized feature gate: Example Nov 04 03:50:02.019145 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003364 6000 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 03:50:02.019145 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003369 6000 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 03:50:02.019145 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003373 6000 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 03:50:02.019145 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003378 6000 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 03:50:02.019145 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003383 6000 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 03:50:02.019145 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003387 6000 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 03:50:02.019145 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003391 6000 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 03:50:02.019145 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003396 6000 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 03:50:02.019145 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003400 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 03:50:02.019145 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003405 6000 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 03:50:02.019145 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003410 6000 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 03:50:02.020051 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003414 6000 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 03:50:02.020051 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003418 6000 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 03:50:02.020051 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003423 6000 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 03:50:02.020051 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003427 6000 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 03:50:02.020051 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003432 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 03:50:02.020051 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003437 6000 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 03:50:02.020051 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003441 6000 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 03:50:02.020051 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003445 6000 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 03:50:02.020051 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003449 6000 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 03:50:02.020051 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003454 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 03:50:02.020051 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003458 6000 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 03:50:02.020051 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003462 6000 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 03:50:02.020051 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003467 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 03:50:02.020051 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003471 6000 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 03:50:02.020051 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003476 6000 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 03:50:02.020051 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003480 6000 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 03:50:02.020051 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003485 6000 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 03:50:02.020051 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003489 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 03:50:02.020051 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003493 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 03:50:02.020933 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003498 6000 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 03:50:02.020933 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003502 6000 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 03:50:02.020933 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003507 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 03:50:02.020933 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003512 6000 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 03:50:02.020933 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003516 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 03:50:02.020933 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003521 6000 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 03:50:02.020933 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003525 6000 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 03:50:02.020933 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003529 6000 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 03:50:02.020933 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003534 6000 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 03:50:02.020933 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003538 6000 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 03:50:02.020933 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003542 6000 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 03:50:02.020933 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003546 6000 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 03:50:02.020933 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003551 6000 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 03:50:02.020933 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003555 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 03:50:02.020933 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003559 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 03:50:02.020933 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003563 6000 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 03:50:02.020933 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003568 6000 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 03:50:02.020933 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003572 6000 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 03:50:02.020933 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003576 6000 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 03:50:02.021806 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003582 6000 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 03:50:02.021806 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003586 6000 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 03:50:02.021806 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003590 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 03:50:02.021806 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003595 6000 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 03:50:02.021806 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003599 6000 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 03:50:02.021806 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003603 6000 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 03:50:02.021806 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003608 6000 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 03:50:02.021806 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003612 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 03:50:02.021806 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003616 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 03:50:02.021806 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003620 6000 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 03:50:02.021806 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003626 6000 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 03:50:02.021806 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003630 6000 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 03:50:02.021806 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003634 6000 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 03:50:02.021806 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003638 6000 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 03:50:02.021806 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003643 6000 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 03:50:02.021806 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003647 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 03:50:02.021806 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003651 6000 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 03:50:02.021806 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003656 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 03:50:02.021806 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003660 6000 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 03:50:02.021806 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003664 6000 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 03:50:02.022694 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003668 6000 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 03:50:02.022694 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003673 6000 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 03:50:02.022694 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003678 6000 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 03:50:02.022694 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003682 6000 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 03:50:02.022694 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003686 6000 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 03:50:02.022694 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003691 6000 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 03:50:02.022694 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003695 6000 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 03:50:02.022694 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003699 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 03:50:02.022694 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003704 6000 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 03:50:02.022694 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003708 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 03:50:02.022694 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003713 6000 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 03:50:02.022694 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003717 6000 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 03:50:02.022694 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003722 6000 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 03:50:02.022694 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003727 6000 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 03:50:02.022694 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003732 6000 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 03:50:02.022694 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.003788 6000 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 03:50:02.023440 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003844 6000 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 03:50:02.023440 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003851 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 03:50:02.023440 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003857 6000 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 03:50:02.023440 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003862 6000 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 03:50:02.023440 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003867 6000 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 03:50:02.023440 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003872 6000 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 03:50:02.023440 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003876 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 03:50:02.023440 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003881 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 03:50:02.023440 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003886 6000 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 03:50:02.023440 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003890 6000 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 03:50:02.023440 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003894 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 03:50:02.023440 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003899 6000 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 03:50:02.023440 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003903 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 03:50:02.023440 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003907 6000 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 03:50:02.023440 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003911 6000 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 03:50:02.023440 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003915 6000 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 03:50:02.023440 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003920 6000 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 03:50:02.023440 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003924 6000 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 03:50:02.023440 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003928 6000 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 03:50:02.023440 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003932 6000 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 03:50:02.024363 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003937 6000 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 03:50:02.024363 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003941 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 03:50:02.024363 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003945 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 03:50:02.024363 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003950 6000 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 03:50:02.024363 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003955 6000 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 03:50:02.024363 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003959 6000 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 03:50:02.024363 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003963 6000 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 03:50:02.024363 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003968 6000 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 03:50:02.024363 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003972 6000 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 03:50:02.024363 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003976 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 03:50:02.024363 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003983 6000 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 03:50:02.024363 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003988 6000 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 03:50:02.024363 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003992 6000 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 03:50:02.024363 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.003997 6000 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 03:50:02.024363 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004005 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 03:50:02.024363 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004009 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 03:50:02.024363 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004013 6000 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 03:50:02.024363 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004018 6000 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 03:50:02.024363 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004023 6000 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 03:50:02.025237 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004027 6000 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 03:50:02.025237 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004032 6000 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 03:50:02.025237 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004037 6000 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 03:50:02.025237 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004041 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 03:50:02.025237 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004045 6000 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 03:50:02.025237 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004050 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 03:50:02.025237 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004054 6000 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 03:50:02.025237 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004059 6000 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 03:50:02.025237 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004063 6000 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 03:50:02.025237 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004067 6000 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 03:50:02.025237 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004072 6000 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 03:50:02.025237 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004076 6000 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 03:50:02.025237 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004080 6000 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 03:50:02.025237 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004085 6000 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 03:50:02.025237 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004089 6000 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 03:50:02.025237 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004093 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 03:50:02.025237 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004098 6000 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 03:50:02.025237 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004102 6000 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 03:50:02.025237 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004106 6000 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 03:50:02.025237 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004111 6000 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 03:50:02.026160 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004115 6000 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 03:50:02.026160 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004119 6000 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 03:50:02.026160 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004124 6000 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 03:50:02.026160 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004128 6000 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 03:50:02.026160 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004136 6000 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 03:50:02.026160 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004144 6000 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 03:50:02.026160 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004150 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 03:50:02.026160 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004158 6000 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 03:50:02.026160 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004163 6000 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 03:50:02.026160 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004169 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 03:50:02.026160 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004174 6000 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 03:50:02.026160 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004178 6000 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 03:50:02.026160 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004182 6000 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 03:50:02.026160 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004187 6000 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 03:50:02.026160 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004192 6000 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 03:50:02.026160 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004196 6000 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 03:50:02.026160 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004200 6000 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 03:50:02.026160 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004205 6000 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 03:50:02.026160 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004210 6000 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 03:50:02.026960 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004214 6000 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 03:50:02.026960 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004219 6000 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 03:50:02.026960 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004224 6000 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 03:50:02.026960 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004228 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 03:50:02.026960 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004232 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 03:50:02.026960 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004236 6000 feature_gate.go:386] unrecognized feature gate: Example Nov 04 03:50:02.026960 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004241 6000 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 03:50:02.026960 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004245 6000 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 03:50:02.026960 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004249 6000 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 03:50:02.026960 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004254 6000 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 03:50:02.026960 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004259 6000 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 03:50:02.026960 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004266 6000 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 03:50:02.026960 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004270 6000 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 03:50:02.026960 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004274 6000 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 03:50:02.026960 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004281 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 03:50:02.026960 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004286 6000 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 03:50:02.026960 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004290 6000 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 03:50:02.026960 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004295 6000 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 03:50:02.026960 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004300 6000 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 03:50:02.026960 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004304 6000 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 03:50:02.027563 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004311 6000 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 03:50:02.027563 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004315 6000 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 03:50:02.027563 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004320 6000 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 03:50:02.027563 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004324 6000 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 03:50:02.027563 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004328 6000 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 03:50:02.027563 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004332 6000 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 03:50:02.027563 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004337 6000 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 03:50:02.027563 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:02.004341 6000 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 03:50:02.027563 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.004440 6000 server.go:957] "Client rotation is on, will bootstrap in background" Nov 04 03:50:02.027563 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.006082 6000 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Nov 04 03:50:02.027563 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.006167 6000 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Nov 04 03:50:02.027563 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.006447 6000 server.go:1014] "Starting client certificate rotation" Nov 04 03:50:02.027563 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.006577 6000 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 03:50:02.027563 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.006630 6000 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 03:43:48 +0000 UTC" deadline="2028-07-20 02:07:46.211804294 +0000 UTC" Nov 04 03:50:02.027563 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.006658 6000 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14974h17m44.20514813s" Nov 04 03:50:02.027563 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.007268 6000 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 03:50:02.028078 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.009027 6000 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 03:50:02.028078 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.010285 6000 log.go:25] "Validated CRI v1 runtime API" Nov 04 03:50:02.028078 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.012202 6000 log.go:25] "Validated CRI v1 image API" Nov 04 03:50:02.028078 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.012991 6000 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Nov 04 03:50:02.028078 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.015126 6000 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7b3a2a3f-ff03-47f0-b018-938c8d5fbee9:/dev/nvme0n1p3 944ac331-65b0-4051-ae09-e6420eb62285:/dev/nvme0n1p4] Nov 04 03:50:02.028343 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.015148 6000 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1694654274d75c17fa3418ca8df7950e1bf8a3fd72c1096b885492cd16ddf7a7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1694654274d75c17fa3418ca8df7950e1bf8a3fd72c1096b885492cd16ddf7a7/userdata/shm major:0 minor:345 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1ab58e8cd93b2dc15d1ce30204828c3d59ecad172948a145590ac93dec61b14d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1ab58e8cd93b2dc15d1ce30204828c3d59ecad172948a145590ac93dec61b14d/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/227320894eae3ad9b870f9978853be7605090c7c784e2642a0c239cc0ed64c68/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/227320894eae3ad9b870f9978853be7605090c7c784e2642a0c239cc0ed64c68/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/23c9921b11f6c392fc6c134e044f2b9c16296d9fea93e4c7e98458ca442cdd8b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/23c9921b11f6c392fc6c134e044f2b9c16296d9fea93e4c7e98458ca442cdd8b/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/252bd07aaa5fd2b939fb6a12717ac49ec8facbc3de2f1218886ad60e56a86545/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/252bd07aaa5fd2b939fb6a12717ac49ec8facbc3de2f1218886ad60e56a86545/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/30acd61540bcb6cb1bdbd8114cac25ad46b938f4c4b6516378cf6ab8c8b3623e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/30acd61540bcb6cb1bdbd8114cac25ad46b938f4c4b6516378cf6ab8c8b3623e/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3a388a934358f893ce03e72f28185b96d6f68ff1d0423739d397198c220537a7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3a388a934358f893ce03e72f28185b96d6f68ff1d0423739d397198c220537a7/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/485212cf3b7bfb927460275c406b97f098b60a53694c113965a2a3cdac372436/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/485212cf3b7bfb927460275c406b97f098b60a53694c113965a2a3cdac372436/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4f15a525e7d518e05acbb46f1a2f234272cc3fbd6461fa94a997007447d95dfa/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4f15a525e7d518e05acbb46f1a2f234272cc3fbd6461fa94a997007447d95dfa/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/51a75d1eecffb2b94dbaec40d2e7567f10f4f26e844f2476fec0bd7357a87e38/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/51a75d1eecffb2b94dbaec40d2e7567f10f4f26e844f2476fec0bd7357a87e38/userdata/shm major:0 minor:362 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5e81bd6796820adb866f5400e7a2075890b09341253469f90fa40cac9a7346be/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5e81bd6796820adb866f5400e7a2075890b09341253469f90fa40cac9a7346be/userdata/shm major:0 minor:271 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6590bc15222877a1cd54c05547b745b132dddd26397761af261ce8473d3d7295/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6590bc15222877a1cd54c05547b745b132dddd26397761af261ce8473d3d7295/userdata/shm major:0 minor:360 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/65a07e471632b7210133b9bc4f9764e3c56e676e8dded2c4ce2b3943e3fddc6b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/65a07e471632b7210133b9bc4f9764e3c56e676e8dded2c4ce2b3943e3fddc6b/userdata/shm major:0 minor:358 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7d0369f4f9735d20f2cb7c87a7d4aa713189f161cfb549cfb9b55f700873604c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7d0369f4f9735d20f2cb7c87a7d4aa713189f161cfb549cfb9b55f700873604c/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7ead774dcedf1f778aa4931d15c7a1dd22b1748a24133f95dac09e4a3f393764/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7ead774dcedf1f778aa4931d15c7a1dd22b1748a24133f95dac09e4a3f393764/userdata/shm major:0 minor:51 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/82178b5fdd0242d2074a69d7b974fbc049b8a4bef0605f94ce7089ec8b5347f6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/82178b5fdd0242d2074a69d7b974fbc049b8a4bef0605f94ce7089ec8b5347f6/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/97588462a6ec62deb01f9d16792c6618b07ca79108ede868df2fb0c74bff5a84/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/97588462a6ec62deb01f9d16792c6618b07ca79108ede868df2fb0c74bff5a84/userdata/shm major:0 minor:284 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b7658efe8fc795d2864a1422877fa92a46b3c3777d4de011a007d01cce63afc3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b7658efe8fc795d2864a1422877fa92a46b3c3777d4de011a007d01cce63afc3/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bbf6bd95a7b883a52463bb9f0aa2c253329afd5c317a40ec8cb5bdf55a17c43f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bbf6bd95a7b883a52463bb9f0aa2c253329afd5c317a40ec8cb5bdf55a17c43f/userdata/shm major:0 minor:307 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c91db1941719fe8a6ca13bb1d85d6b0fd162b50b3f18cfa0e6aa49aa1d331aa6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c91db1941719fe8a6ca13bb1d85d6b0fd162b50b3f18cfa0e6aa49aa1d331aa6/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e6265225ad2d7fd61a17e7506274cd6ee39345729f55043c8c0d1796c9f1d66a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e6265225ad2d7fd61a17e7506274cd6ee39345729f55043c8c0d1796c9f1d66a/userdata/shm major:0 minor:230 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06a274ba-93f4-4dbc-ac39-4039fd254e7d/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/06a274ba-93f4-4dbc-ac39-4039fd254e7d/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:356 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07/volumes/kubernetes.io~projected/kube-api-access-n49rr:{mountpoint:/var/lib/kubelet/pods/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07/volumes/kubernetes.io~projected/kube-api-access-n49rr major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2502af7a-1118-4c5e-84b8-8c97188c09a4/volumes/kubernetes.io~projected/kube-api-access-bv8hw:{mountpoint:/var/lib/kubelet/pods/2502af7a-1118-4c5e-84b8-8c97188c09a4/volumes/kubernetes.io~projected/kube-api-access-bv8hw major:0 minor:236 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2502af7a-1118-4c5e-84b8-8c97188c09a4/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/2502af7a-1118-4c5e-84b8-8c97188c09a4/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config major:0 minor:235 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2502af7a-1118-4c5e-84b8-8c97188c09a4/volumes/kubernetes.io~secret/prometheus-operator-tls:{mountpoint:/var/lib/kubelet/pods/2502af7a-1118-4c5e-84b8-8c97188c09a4/volumes/kubernetes.io~secret/prometheus-operator-tls major:0 minor:234 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/27390c92-0480-4b55-ba3d-e20f7ada5b83/volumes/kubernetes.io~projected/kube-api-access-gf87g:{mountpoint:/var/lib/kubelet/pods/27390c92-0480-4b55-ba3d-e20f7ada5b83/volumes/kubernetes.io~projected/kube-api-access-gf87g major:0 minor:297 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/27390c92-0480-4b55-ba3d-e20f7ada5b83/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/27390c92-0480-4b55-ba3d-e20f7ada5b83/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:296 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2b70504f-4169-44b2-8b4e-979a068f070d/volumes/kubernetes.io~projected/kube-api-access-txw7k:{mountpoint:/var/lib/kubelet/pods/2b70504f-4169-44b2-8b4e-979a068f070d/volumes/kubernetes.io~projected/kube-api-access-txw7k major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3e07cc7a-88c6-4e29-b7ab-c078916cd977/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/3e07cc7a-88c6-4e29-b7ab-c078916cd977/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3e07cc7a-88c6-4e29-b7ab-c078916cd977/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/3e07cc7a-88c6-4e29-b7ab-c078916cd977/volumes/kubernetes.io~empty-dir/tmp major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3e07cc7a-88c6-4e29-b7ab-c078916cd977/volumes/kubernetes.io~projected/kube-api-access-4b9rr:{mountpoint:/var/lib/kubelet/pods/3e07cc7a-88c6-4e29-b7ab-c078916cd977/volumes/kubernetes.io~projected/kube-api-access-4b9rr major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/42510c3d-0283-4901-9d23-a84793598ac9/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/42510c3d-0283-4901-9d23-a84793598ac9/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/42510c3d-0283-4901-9d23-a84793598ac9/volumes/kubernetes.io~projected/kube-api-access-9vpwl:{mountpoint:/var/lib/kubelet/pods/42510c3d-0283-4901-9d23-a84793598ac9/volumes/kubernetes.io~projected/kube-api-access-9vpwl major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/42510c3d-0283-4901-9d23-a84793598ac9/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/42510c3d-0283-4901-9d23-a84793598ac9/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb/volumes/kubernetes.io~projected/kube-api-access-4zpwv:{mountpoint:/var/lib/kubelet/pods/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb/volumes/kubernetes.io~projected/kube-api-access-4zpwv major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/56348938-ef44-41ab-8a71-acc1e4308262/volumes/kubernetes.io~projected/kube-api-access-94ws5:{mountpoint:/var/lib/kubelet/pods/56348938-ef44-41ab-8a71-acc1e4308262/volumes/kubernetes.io~projected/kube-api-access-94ws5 major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/56348938-ef44-41ab-8a71-acc1e4308262/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/56348938-ef44-41ab-8a71-acc1e4308262/volumes/kubernetes.io~secret/metrics-certs major:0 minor:351 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5bdb7796-b1a3-49a5-81cd-c8101320e172/volumes/kubernetes.io~projected/kube-api-access-68d54:{mountpoint:/var/lib/kubelet/pods/5bdb7796-b1a3-49a5-81cd-c8101320e172/volumes/kubernetes.io~projected/kube-api-access-68d54 major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5cf7e388-1678-4afb-8877-30179162401b/volumes/kubernetes.io~projected/kube-api-access-8lvzv:{mountpoint:/var/lib/kubelet/pods/5cf7e388-1678-4afb-8877-30179162401b/volumes/kubernetes.io~projected/kube-api-access-8lvzv major:0 minor:357 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a6918a9-3ff2-46aa-bc91-affd1835403d/volumes/kubernetes.io~projected/kube-api-access-hjqhd:{mountpoint:/var/lib/kubelet/pods/8a6918a9-3ff2-46aa-bc91-affd1835403d/volumes/kubernetes.io~projected/kube-api-access-hjqhd major:0 minor:281 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a6918a9-3ff2-46aa-bc91-affd1835403d/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/8a6918a9-3ff2-46aa-bc91-affd1835403d/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:275 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a6918a9-3ff2-46aa-bc91-affd1835403d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/8a6918a9-3ff2-46aa-bc91-affd1835403d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:276 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a6918a9-3ff2-46aa-bc91-affd1835403d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/8a6918a9-3ff2-46aa-bc91-affd1835403d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:278 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a6918a9-3ff2-46aa-bc91-affd1835403d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/8a6918a9-3ff2-46aa-bc91-affd1835403d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:277 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a6918a9-3ff2-46aa-bc91-affd1835403d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/8a6918a9-3ff2-46aa-bc91-affd1835403d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:280 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a6918a9-3ff2-46aa-bc91-affd1835403d/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/8a6918a9-3ff2-46aa-bc91-affd1835403d/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:279 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/94c06e7a-0510-4787-8b60-45b185300091/volumes/kubernetes.io~projected/kube-api-access-62d4x:{mountpoint:/var/lib/kubelet/pods/94c06e7a-0510-4787-8b60-45b185300091/volumes/kubernetes.io~projected/kube-api-access-62d4x major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/94c06e7a-0510-4787-8b60-45b185300091/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/94c06e7a-0510-4787-8b60-45b185300091/volumes/kubernetes.io~secret/metrics-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b1091f60-668b-4884-8548-fe4f4c438367/volumes/kubernetes.io~projected/kube-api-access-bgmb8:{mountpoint:/var/lib/kubelet/pods/b1091f60-668b-4884-8548-fe4f4c438367/volumes/kubernetes.io~projected/kube-api-access-bgmb8 major:0 minor:270 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b1091f60-668b-4884-8548-fe4f4c438367/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/b1091f60-668b-4884-8548-fe4f4c438367/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:269 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b1091f60-668b-4884-8548-fe4f4c438367/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/b1091f60-668b-4884-8548-fe4f4c438367/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:265 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dde9ca65-5089-450e-97cd-89bf66adba30/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/dde9ca65-5089-450e-97cd-89bf66adba30/volumes/kubernetes.io~secret/agent-certs major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8/volumes/kubernetes.io~projected/kube-api-access-gkcvz:{mountpoint:/var/lib/kubelet/pods/e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8/volumes/kubernetes.io~projected/kube-api-access-gkcvz major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volume-subpaths/web-config/config-reloader/2:{mountpoint:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volume-subpaths/web-config/config-reloader/2 major:0 minor:342 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volume-subpaths/web-config/init-config-reloader/2:{mountpoint:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volume-subpaths/web-config/init-config-reloader/2 major:0 minor:342 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volume-subpaths/web-config/prometheus/15:{mountpoint:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volume-subpaths/web-config/prometheus/15 major:0 minor:342 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~empty-dir/config-out major:0 minor:336 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~projected/kube-api-access-zlw9q:{mountpoint:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~projected/kube-api-access-zlw9q major:0 minor:343 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~projected/tls-assets major:0 minor:338 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/config:{mountpoint:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/config major:0 minor:330 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:335 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/secret-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/secret-kube-rbac-proxy major:0 minor:344 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/secret-metrics-client-certs:{mountpoint:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/secret-metrics-client-certs major:0 minor:340 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web major:0 minor:339 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls:{mountpoint:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls major:0 minor:341 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls:{mountpoint:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls major:0 minor:334 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file:{mountpoint:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file major:0 minor:337 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/web-config major:0 minor:342 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ea2d0497-b79b-4640-b8ca-bd829b1f3dfa/volumes/kubernetes.io~projected/kube-api-access-jm29k:{mountpoint:/var/lib/kubelet/pods/ea2d0497-b79b-4640-b8ca-bd829b1f3dfa/volumes/kubernetes.io~projected/kube-api-access-jm29k major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ea2d0497-b79b-4640-b8ca-bd829b1f3dfa/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/ea2d0497-b79b-4640-b8ca-bd829b1f3dfa/volumes/kubernetes.io~secret/cert major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eaec1015-2b2b-4f06-b41d-028671919cf0/volumes/kubernetes.io~projected/kube-api-access-ftm9l:{mountpoint:/var/lib/kubelet/pods/eaec1015-2b2b-4f06-b41d-028671919cf0/volumes/kubernetes.io~projected/kube-api-access-ftm9l major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6/volumes/kubernetes.io~projected/kube-api-access-z7hkr:{mountpoint:/var/lib/kubelet/pods/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6/volumes/kubernetes.io~projected/kube-api-access-z7hkr major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:201 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/790efcde84b1d57c33424b4887aa40c51579af27d0c140f05d5bcaafd7a0fe55/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/c9c8176e8977d08261ab5ee12f0af13ac6accbb2fd8ae120b2e815b673150fd9/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/4758840f368e97cabec5e3f5a2c57d037d0428be391003f0ce948f5a71f20ab6/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/7681654cad9ca012c95dd0ee6b76110a78eb501be050e945e55db269e42fe8a1/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/f7ebc60ab7161452683d8076b6968f16824c3bb0160dd734d51b4b9812285c92/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/024a05d5dea5fa48376667dab75a8646a91351875efe96cfddf3aea89fa634f4/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/2aa184f5f88e9f72d5c4ad943f61ecac20baa98f02d7a61f0ada949fb8246002/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/e4a33118c2e4816730fad4e1b2a603f91966c550083d8ff361e583f869ec177d/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-128:{mountpoint:/var/lib/containers/storage/overlay/2782ed68da13ab7a790c4d3f9e4494d97026c6e8181e1d710c69bc43489e099f/merged major:0 minor:128 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/6dd5f9d4d9d7bcd74f167819137e06240e8ea6b030d4cde77e29143747fbc76a/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-142:{mountpoint:/var/lib/containers/storage/overlay/3a5198269b7a860a611480ca6292d3d09950171e83d38e90f57819d2600228fc/merged major:0 minor:142 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/14288db07b2e27458bb42cd6955e2a6ce6298d54a2e86b36663cf28a20d57f54/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/3db7e629ddf7c4944676f772f676996fb93b8a77333a12177a92f99a2f0a254d/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-154:{mountpoint:/var/lib/containers/storage/overlay/b7159805a4eb8706c5ac6daea40452f69fa42f20f5fa7c990f61cdd13e238de9/merged major:0 minor:154 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/395cd6589f69533769334afebb05de0a58368d151e85ff076f63650f9dfc529d/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/61a5b7453a257e2a71539b5261abaf86383d7a547360c3fd375d2efc95a4be22/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/e2ddfdcdca525a1307e9236fb5e1839618a51578423bc8a08b531477a0411168/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/d3143e1092595864524a29e3aa6354911e021a9614cdc9bad515015dea74bf23/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/8b6bde17030b64fa04717ba0391902fc576f75c33da339e5fa1f0f9c7766e3f2/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/4d20b57c05f213e36a8c1f10449956a62f7ae654989ade230d9b63c21e08de3d/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/2a34c0bb590dd732ec2a1fcbd65897614192921ed0760fa0fa8bd2eeb645d6c1/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/402dc0f6a2dbe626b6a7c7a5416858fedf48fb64e0255fe36501d22f40a0aaa8/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/df32a3e47ea0809a07f26be625de9bf4a9ddc1c0661e69caea90d23a8c44b84e/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/421908e8470be1fa1a62886ac98053418f0c4ed5352aa7e181f3b24f6932e575/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/2a2672bd2ac80b06028a5e31ec151103055fc42f627ba6de1a5c79d798abd604/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/75942b258fab485a6605778523db190238b69819ed7bce34158154816bf8a51f/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/0ad1a905dec89822085a7845d2298d44fe4530f2d38c67cf4d0c8cda95557a6c/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-238:{mountpoint:/var/lib/containers/storage/overlay/1bb58f026c0402040643e5ff586e5d433df11de893829ef531e5babe5b21c37d/merged major:0 minor:238 fsType:overlay blockSize:0} overlay_0-240:{mountpoint:/var/lib/containers/storage/overlay/794fad59083ab7f6c20dd6030d9865d20300c02f1da5b8a8de021b5287e12afc/merged major:0 minor:240 fsType:overlay blockSize:0} overlay_0-242:{mountpoint:/var/lib/containers/storage/overlay/af0621c2de215954f324fd044376be52bca9584e6760ddf019f7699fdf573b6e/merged major:0 minor:242 fsType:overlay blockSize:0} overlay_0-244:{mountpoint:/var/lib/containers/storage/overlay/08fb932288044f3baad40266a2f77855218b3f91d60217469217a949545a4809/merged major:0 minor:244 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/9d2852d534da5980d1ca51e524aa3c0c38650e97d84b93118b8f3eae5c80e57a/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-259:{mountpoint:/var/lib/containers/storage/overlay/bf10339483831ee2b76189e67fd9cdeba6ac08ecc55a823e7703e5167a179da5/merged major:0 minor:259 fsType:overlay blockSize:0} overlay_0-273:{mountpoint:/var/lib/containers/storage/overlay/87a67701d0aca6e110d4318cb7df36f91857a5acb2494670b10e01c814a54a6a/merged major:0 minor:273 fsType:overlay blockSize:0} overlay_0-282:{mountpoint:/var/lib/containers/storage/overlay/a9f08d91b782d8c8523f10927c1f2b201ef53804508bf4feb9e57709df8f9fd3/merged major:0 minor:282 fsType:overlay blockSize:0} overlay_0-286:{mountpoint:/var/lib/containers/storage/overlay/1dfb7b73eed71546c1e1e191773b99fbdb7c000541e6d295d33b672d37507950/merged major:0 minor:286 fsType:overlay blockSize:0} overlay_0-291:{mountpoint:/var/lib/containers/storage/overlay/dda3403a77e4f98702456050b55d1996685a79358384839ffecee30b4fcb115d/merged major:0 minor:291 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/505d80de384e289ab86ac7b243eff1b483449f045dacc8f642dc1ac2face4b44/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/10c6450e859d048e476a0bb53ae1407284c4d0be36a55bc274ec737fefa5ac40/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-306:{mountpoint:/var/lib/containers/storage/overlay/bbf163a9a314e2c186edf12454270fa0e4695b0af60df7394cae0fc5869099c4/merged major:0 minor:306 fsType:overlay blockSize:0} overlay_0-313:{mountpoint:/var/lib/containers/storage/overlay/ef9938947c7cb556fa0e31385a08b88884c46fbe4e5e4de832f90bc36b372424/merged major:0 minor:313 fsType:overlay blockSize:0} overlay_0-318:{mountpoint:/var/lib/containers/storage/overlay/c52892e9f95d9de1f510ec01e7764012d68c21ac9087929f91f03150f90a64c8/merged major:0 minor:318 fsType:overlay blockSize:0} overlay_0-320:{mountpoint:/var/lib/containers/storage/overlay/3e68921914724c5170719c55743b8e56c2a302b913582ace8e9c396536424181/merged major:0 minor:320 fsType:overlay blockSize:0} overlay_0-325:{mountpoint:/var/lib/containers/storage/overlay/14d89684091aa5a5cd06dc8e748bc154c0fef94ca661cd388f676a63255635fe/merged major:0 minor:325 fsType:overlay blockSize:0} overlay_0-347:{mountpoint:/var/lib/containers/storage/overlay/d02b5d6afbd411f67721ce453af5889c3e2f47c47f322d4a9c2d784398859322/merged major:0 minor:347 fsType:overlay blockSize:0} overlay_0-349:{mountpoint:/var/lib/containers/storage/overlay/87488923ec0c3bf746d78ac32042994bc7c1fb3eee20c851a59773fbb65fa046/merged major:0 minor:349 fsType:overlay blockSize:0} overlay_0-364:{mountpoint:/var/lib/containers/storage/overlay/8e3df983ccb1c8fb236b133e2a82838bce4ec5851d743118df6ce716810cdd40/merged major:0 minor:364 fsType:overlay blockSize:0} overlay_0-366:{mountpoint:/var/lib/containers/storage/overlay/ba68568f89b93e9f6275fd0bd48caf182b4986930950fab32e56be83d82ceab1/merged major:0 minor:366 fsType:overlay blockSize:0} overlay_0-368:{mountpoint:/var/lib/containers/storage/overlay/38944b5342ce9a274a7e65db0a40c9dfa201838d03fbd73ca553775f78649b98/merged major:0 minor:368 fsType:overlay blockSize:0} overlay_0-370:{mountpoint:/var/lib/containers/storage/overlay/a18fa2ab3c6d203c59285484af9b8ba62c4faaf76707ca0455ac3a90816e4abf/merged major:0 minor:370 fsType:overlay blockSize:0} overlay_0-372:{mountpoint:/var/lib/containers/storage/overlay/3931a33316abd90ec4aa7164e5edce7f9151d085a509942c7820b13a67e7cd4a/merged major:0 minor:372 fsType:overlay blockSize:0} overlay_0-378:{mountpoint:/var/lib/containers/storage/overlay/d0d57542442988ac13ccf1bbbca3f5df9fb86a789680026ffa9164d8f714fb18/merged major:0 minor:378 fsType:overlay blockSize:0} overlay_0-383:{mountpoint:/var/lib/containers/storage/overlay/791406d4725fa5a6710443aae4b4ed85f777bc3736ab4e09ce84d04bf81534bd/merged major:0 minor:383 fsType:overlay blockSize:0} overlay_0-389:{mountpoint:/var/lib/containers/storage/overlay/07af9470d43c6a316e76284ea2a2598929c91e08c81df1636bfed54c1bf857f9/merged major:0 minor:389 fsType:overlay blockSize:0} overlay_0-394:{mountpoint:/var/lib/containers/storage/overlay/35e8b4f0e2618ec5752b432050b69982080801f7381f45b6b5ea2ede62caee5a/merged major:0 minor:394 fsType:overlay blockSize:0} overlay_0-399:{mountpoint:/var/lib/containers/storage/overlay/d0818aa6b9df57b412372dfdc757ffb65a482e1530edacf24fdf0135aaea20d4/merged major:0 minor:399 fsType:overlay blockSize:0} overlay_0-404:{mountpoint:/var/lib/containers/storage/overlay/b412407939196d281e1fa8d81e98ca40b73590503488ef69b28be84698b3e52d/merged major:0 minor:404 fsType:overlay blockSize:0} overlay_0-409:{mountpoint:/var/lib/containers/storage/overlay/58f7eb793d39c4ed84b24bf7cbe57e5e39ca049bc4c3ab6eba4e8a462ec215cb/merged major:0 minor:409 fsType:overlay blockSize:0} overlay_0-414:{mountpoint:/var/lib/containers/storage/overlay/bcdf671ab7621f00d8ed18b8e5928601baf018a53c7d6ba0d733b8cad3d240a5/merged major:0 minor:414 fsType:overlay blockSize:0} overlay_0-49:{mountpoint:/var/lib/containers/storage/overlay/c59f0946b7302f245d0abb0beb38c7359c202c7ce1ed1ed6e291fdd1bb7b4cc0/merged major:0 minor:49 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/e143eca7c4bfe9696a6cf31c129b5cb7ed2220025b49fbf8943d936eabd10ca9/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/04d23998f50e8df4b6d763caefd596dae5b23e6765e52fb03b76986abb54ca6b/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/d5d1c3c1dc5a9e92073f57fb5be5f06af154c3c000bfe15d56d06aedbfd5462d/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/a0e79b6561dfbe552d1d0e05f74db8446c3dca22620fc15d1da67bf3e1c3a8db/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/f8c42941a76ba6921f04e3d959771d9e38d4a46adef803f4b5a1d92a6984d689/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/76410eb92f5b6a26514f430c524fe16f84646b9b7de7291a7960972399de14a3/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/0b291dec849df1c1c3d20d44cc4cadafeb98b6a0a673c103b28616db0ff8a903/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/c5aae10ffff42e75b5223f0414e1890505cc77b41c632b9d7df219dd94178d59/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/b1721a8a9c33d5ab8b28abb923890a616f1be4828e11ba021fe163d4cc59181e/merged major:0 minor:96 fsType:overlay blockSize:0}] Nov 04 03:50:02.028727 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.026016 6000 manager.go:217] Machine: {Timestamp:2026-11-04 03:50:02.02519654 +0000 UTC m=+0.063636507 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101936 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20bee300a0e6f49cecca986af8fa37 SystemUUID:ec20bee3-00a0-e6f4-9cec-ca986af8fa37 BootID:40862e3d-e54b-4230-96be-d63a9aff4b15 Filesystems:[{Device:/var/lib/kubelet/pods/42510c3d-0283-4901-9d23-a84793598ac9/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07/volumes/kubernetes.io~projected/kube-api-access-n49rr DeviceMajor:0 DeviceMinor:65 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/227320894eae3ad9b870f9978853be7605090c7c784e2642a0c239cc0ed64c68/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8a6918a9-3ff2-46aa-bc91-affd1835403d/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:279 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-282 DeviceMajor:0 DeviceMinor:282 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-313 DeviceMajor:0 DeviceMinor:313 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-325 DeviceMajor:0 DeviceMinor:325 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/2b70504f-4169-44b2-8b4e-979a068f070d/volumes/kubernetes.io~projected/kube-api-access-txw7k DeviceMajor:0 DeviceMinor:57 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6/volumes/kubernetes.io~projected/kube-api-access-z7hkr DeviceMajor:0 DeviceMinor:204 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-244 DeviceMajor:0 DeviceMinor:244 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5e81bd6796820adb866f5400e7a2075890b09341253469f90fa40cac9a7346be/userdata/shm DeviceMajor:0 DeviceMinor:271 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ea2d0497-b79b-4640-b8ca-bd829b1f3dfa/volumes/kubernetes.io~projected/kube-api-access-jm29k DeviceMajor:0 DeviceMinor:202 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-240 DeviceMajor:0 DeviceMinor:240 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-259 DeviceMajor:0 DeviceMinor:259 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:336 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-409 DeviceMajor:0 DeviceMinor:409 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/56348938-ef44-41ab-8a71-acc1e4308262/volumes/kubernetes.io~projected/kube-api-access-94ws5 DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8a6918a9-3ff2-46aa-bc91-affd1835403d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:276 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8a6918a9-3ff2-46aa-bc91-affd1835403d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:280 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb/volumes/kubernetes.io~projected/kube-api-access-4zpwv DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b7658efe8fc795d2864a1422877fa92a46b3c3777d4de011a007d01cce63afc3/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-414 DeviceMajor:0 DeviceMinor:414 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/82178b5fdd0242d2074a69d7b974fbc049b8a4bef0605f94ce7089ec8b5347f6/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b1091f60-668b-4884-8548-fe4f4c438367/volumes/kubernetes.io~projected/kube-api-access-bgmb8 DeviceMajor:0 DeviceMinor:270 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1694654274d75c17fa3418ca8df7950e1bf8a3fd72c1096b885492cd16ddf7a7/userdata/shm DeviceMajor:0 DeviceMinor:345 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volume-subpaths/web-config/init-config-reloader/2 DeviceMajor:0 DeviceMinor:342 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-49 DeviceMajor:0 DeviceMinor:49 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8/volumes/kubernetes.io~projected/kube-api-access-gkcvz DeviceMajor:0 DeviceMinor:64 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c91db1941719fe8a6ca13bb1d85d6b0fd162b50b3f18cfa0e6aa49aa1d331aa6/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ea2d0497-b79b-4640-b8ca-bd829b1f3dfa/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls DeviceMajor:0 DeviceMinor:341 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-383 DeviceMajor:0 DeviceMinor:383 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/485212cf3b7bfb927460275c406b97f098b60a53694c113965a2a3cdac372436/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/94c06e7a-0510-4787-8b60-45b185300091/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b1091f60-668b-4884-8548-fe4f4c438367/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:265 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-349 DeviceMajor:0 DeviceMinor:349 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:335 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file DeviceMajor:0 DeviceMinor:337 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/secret-metrics-client-certs DeviceMajor:0 DeviceMinor:340 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3a388a934358f893ce03e72f28185b96d6f68ff1d0423739d397198c220537a7/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-128 DeviceMajor:0 DeviceMinor:128 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:339 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-368 DeviceMajor:0 DeviceMinor:368 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volume-subpaths/web-config/config-reloader/2 DeviceMajor:0 DeviceMinor:342 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7d0369f4f9735d20f2cb7c87a7d4aa713189f161cfb549cfb9b55f700873604c/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-238 DeviceMajor:0 DeviceMinor:238 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-389 DeviceMajor:0 DeviceMinor:389 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-399 DeviceMajor:0 DeviceMinor:399 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/eaec1015-2b2b-4f06-b41d-028671919cf0/volumes/kubernetes.io~projected/kube-api-access-ftm9l DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bbf6bd95a7b883a52463bb9f0aa2c253329afd5c317a40ec8cb5bdf55a17c43f/userdata/shm DeviceMajor:0 DeviceMinor:307 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/65a07e471632b7210133b9bc4f9764e3c56e676e8dded2c4ce2b3943e3fddc6b/userdata/shm DeviceMajor:0 DeviceMinor:358 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-366 DeviceMajor:0 DeviceMinor:366 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/config DeviceMajor:0 DeviceMinor:330 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-404 DeviceMajor:0 DeviceMinor:404 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8a6918a9-3ff2-46aa-bc91-affd1835403d/volumes/kubernetes.io~projected/kube-api-access-hjqhd DeviceMajor:0 DeviceMinor:281 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-320 DeviceMajor:0 DeviceMinor:320 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5bdb7796-b1a3-49a5-81cd-c8101320e172/volumes/kubernetes.io~projected/kube-api-access-68d54 DeviceMajor:0 DeviceMinor:67 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-242 DeviceMajor:0 DeviceMinor:242 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:338 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/30acd61540bcb6cb1bdbd8114cac25ad46b938f4c4b6516378cf6ab8c8b3623e/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/dde9ca65-5089-450e-97cd-89bf66adba30/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/42510c3d-0283-4901-9d23-a84793598ac9/volumes/kubernetes.io~projected/kube-api-access-9vpwl DeviceMajor:0 DeviceMinor:63 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-142 DeviceMajor:0 DeviceMinor:142 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-306 DeviceMajor:0 DeviceMinor:306 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls DeviceMajor:0 DeviceMinor:334 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:342 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8a6918a9-3ff2-46aa-bc91-affd1835403d/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:275 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/97588462a6ec62deb01f9d16792c6618b07ca79108ede868df2fb0c74bff5a84/userdata/shm DeviceMajor:0 DeviceMinor:284 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-286 DeviceMajor:0 DeviceMinor:286 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-154 DeviceMajor:0 DeviceMinor:154 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5cf7e388-1678-4afb-8877-30179162401b/volumes/kubernetes.io~projected/kube-api-access-8lvzv DeviceMajor:0 DeviceMinor:357 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e6265225ad2d7fd61a17e7506274cd6ee39345729f55043c8c0d1796c9f1d66a/userdata/shm DeviceMajor:0 DeviceMinor:230 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8a6918a9-3ff2-46aa-bc91-affd1835403d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:277 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-378 DeviceMajor:0 DeviceMinor:378 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-394 DeviceMajor:0 DeviceMinor:394 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1ab58e8cd93b2dc15d1ce30204828c3d59ecad172948a145590ac93dec61b14d/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/42510c3d-0283-4901-9d23-a84793598ac9/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:60 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/27390c92-0480-4b55-ba3d-e20f7ada5b83/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:296 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volume-subpaths/web-config/prometheus/15 DeviceMajor:0 DeviceMinor:342 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/3e07cc7a-88c6-4e29-b7ab-c078916cd977/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:61 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3e07cc7a-88c6-4e29-b7ab-c078916cd977/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:62 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2502af7a-1118-4c5e-84b8-8c97188c09a4/volumes/kubernetes.io~secret/prometheus-operator-tls DeviceMajor:0 DeviceMinor:234 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3e07cc7a-88c6-4e29-b7ab-c078916cd977/volumes/kubernetes.io~projected/kube-api-access-4b9rr DeviceMajor:0 DeviceMinor:66 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2502af7a-1118-4c5e-84b8-8c97188c09a4/volumes/kubernetes.io~projected/kube-api-access-bv8hw DeviceMajor:0 DeviceMinor:236 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b1091f60-668b-4884-8548-fe4f4c438367/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:269 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-273 DeviceMajor:0 DeviceMinor:273 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/27390c92-0480-4b55-ba3d-e20f7ada5b83/volumes/kubernetes.io~projected/kube-api-access-gf87g DeviceMajor:0 DeviceMinor:297 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~projected/kube-api-access-zlw9q DeviceMajor:0 DeviceMinor:343 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-372 DeviceMajor:0 DeviceMinor:372 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/51a75d1eecffb2b94dbaec40d2e7567f10f4f26e844f2476fec0bd7357a87e38/userdata/shm DeviceMajor:0 DeviceMinor:362 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-364 DeviceMajor:0 DeviceMinor:364 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4f15a525e7d518e05acbb46f1a2f234272cc3fbd6461fa94a997007447d95dfa/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/23c9921b11f6c392fc6c134e044f2b9c16296d9fea93e4c7e98458ca442cdd8b/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-291 DeviceMajor:0 DeviceMinor:291 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/252bd07aaa5fd2b939fb6a12717ac49ec8facbc3de2f1218886ad60e56a86545/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/94c06e7a-0510-4787-8b60-45b185300091/volumes/kubernetes.io~projected/kube-api-access-62d4x DeviceMajor:0 DeviceMinor:203 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-318 DeviceMajor:0 DeviceMinor:318 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes/kubernetes.io~secret/secret-kube-rbac-proxy DeviceMajor:0 DeviceMinor:344 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/06a274ba-93f4-4dbc-ac39-4039fd254e7d/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:356 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2502af7a-1118-4c5e-84b8-8c97188c09a4/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:235 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-347 DeviceMajor:0 DeviceMinor:347 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8a6918a9-3ff2-46aa-bc91-affd1835403d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:278 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/56348938-ef44-41ab-8a71-acc1e4308262/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:351 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6590bc15222877a1cd54c05547b745b132dddd26397761af261ce8473d3d7295/userdata/shm DeviceMajor:0 DeviceMinor:360 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-370 DeviceMajor:0 DeviceMinor:370 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7ead774dcedf1f778aa4931d15c7a1dd22b1748a24133f95dac09e4a3f393764/userdata/shm DeviceMajor:0 DeviceMinor:51 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:1694654274d75c1 MacAddress:42:8c:4f:ba:9b:5c Speed:10000 Mtu:8901} {Name:227320894eae3ad MacAddress:d2:8c:ab:88:4e:1d Speed:10000 Mtu:8901} {Name:23c9921b11f6c39 MacAddress:1e:b4:74:24:da:07 Speed:10000 Mtu:8901} {Name:51a75d1eecffb2b MacAddress:ca:ef:64:ad:c6:f9 Speed:10000 Mtu:8901} {Name:6590bc15222877a MacAddress:36:7b:2f:9e:e8:d5 Speed:10000 Mtu:8901} {Name:65a07e471632b72 MacAddress:aa:94:d2:49:9f:16 Speed:10000 Mtu:8901} {Name:7d0369f4f9735d2 MacAddress:62:db:64:10:6a:0b Speed:10000 Mtu:8901} {Name:97588462a6ec62d MacAddress:d6:4c:54:ad:43:b0 Speed:10000 Mtu:8901} {Name:bbf6bd95a7b883a MacAddress:16:90:33:5e:5f:fe Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:eb:18:42:4f Speed:0 Mtu:9001} {Name:br-int MacAddress:d2:22:fe:08:a1:b4 Speed:0 Mtu:8901} {Name:e6265225ad2d7fd MacAddress:02:79:85:8e:cc:c7 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:eb:18:42:4f Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:da:83:4f:34:58:01 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:c2:33:b6:44:96:0b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Nov 04 03:50:02.028727 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.026800 6000 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Nov 04 03:50:02.028727 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.026874 6000 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Nov 04 03:50:02.028727 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.027198 6000 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Nov 04 03:50:02.028727 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.027224 6000 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-134.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Nov 04 03:50:02.028727 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.027360 6000 topology_manager.go:143] "Creating topology manager with none policy" Nov 04 03:50:02.028727 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.027369 6000 container_manager_linux.go:311] "Creating device plugin manager" Nov 04 03:50:02.028727 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.027379 6000 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 03:50:02.028727 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.027400 6000 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 03:50:02.028727 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.027513 6000 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Nov 04 03:50:02.028727 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.027679 6000 state_mem.go:41] "Initialized" logger="CPUManager state memory" Nov 04 03:50:02.028727 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.027807 6000 server.go:1235] "Using root directory" path="/var/lib/kubelet" Nov 04 03:50:02.028727 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.027842 6000 kubelet.go:493] "Attempting to sync node with API server" Nov 04 03:50:02.028727 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.027852 6000 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Nov 04 03:50:02.028727 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.027862 6000 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Nov 04 03:50:02.028727 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.027871 6000 kubelet.go:405] "Adding apiserver pod source" Nov 04 03:50:02.028727 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.027879 6000 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Nov 04 03:50:02.028727 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.028503 6000 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Nov 04 03:50:02.028727 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.028513 6000 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 03:50:02.029908 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.029101 6000 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Nov 04 03:50:02.031531 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.031509 6000 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Nov 04 03:50:02.031531 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.031524 6000 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 03:50:02.031670 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.031633 6000 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Nov 04 03:50:02.031865 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.031851 6000 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Nov 04 03:50:02.031898 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.031882 6000 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Nov 04 03:50:02.032024 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.032014 6000 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Nov 04 03:50:02.032068 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.032029 6000 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Nov 04 03:50:02.032068 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.032035 6000 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Nov 04 03:50:02.032068 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.032041 6000 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Nov 04 03:50:02.032068 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.032047 6000 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Nov 04 03:50:02.032068 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.032053 6000 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Nov 04 03:50:02.032068 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.032059 6000 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Nov 04 03:50:02.032068 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.032065 6000 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Nov 04 03:50:02.032310 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.032073 6000 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Nov 04 03:50:02.032310 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.032080 6000 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Nov 04 03:50:02.032310 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.032095 6000 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Nov 04 03:50:02.032310 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.032105 6000 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Nov 04 03:50:02.032310 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.032137 6000 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Nov 04 03:50:02.032310 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.032143 6000 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Nov 04 03:50:02.035613 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.035598 6000 server.go:1263] "Started kubelet" Nov 04 03:50:02.035698 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.035670 6000 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Nov 04 03:50:02.035940 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.035766 6000 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Nov 04 03:50:02.035940 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.035852 6000 server_v1.go:49] "podresources" method="list" useActivePods=true Nov 04 03:50:02.036180 ip-10-0-129-134 systemd[1]: Started Kubernetes Kubelet. Nov 04 03:50:02.037399 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.037357 6000 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Nov 04 03:50:02.041114 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.039143 6000 server.go:317] "Adding debug handlers to kubelet server" Nov 04 03:50:02.041735 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.041549 6000 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Nov 04 03:50:02.041735 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.041571 6000 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Nov 04 03:50:02.041735 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.041697 6000 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 03:43:50 +0000 UTC" deadline="2028-06-27 05:49:46.667992325 +0000 UTC" Nov 04 03:50:02.041735 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.041724 6000 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14425h59m44.6262711s" Nov 04 03:50:02.041735 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.041732 6000 volume_manager.go:309] "The desired_state_of_world populator starts" Nov 04 03:50:02.042050 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.041745 6000 volume_manager.go:311] "Starting Kubelet Volume Manager" Nov 04 03:50:02.042050 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:50:02.041805 6000 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-134.us-east-2.compute.internal\" not found" Nov 04 03:50:02.042050 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.042012 6000 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Nov 04 03:50:02.044599 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044561 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e191ac4e-0058-44af-b530-5e68a054d946" volumeName="kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-2" seLinuxMountContext="" Nov 04 03:50:02.044599 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044596 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e191ac4e-0058-44af-b530-5e68a054d946" volumeName="kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-config" seLinuxMountContext="" Nov 04 03:50:02.044723 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044606 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e191ac4e-0058-44af-b530-5e68a054d946" volumeName="kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-metrics-client-ca" seLinuxMountContext="" Nov 04 03:50:02.044723 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044615 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e191ac4e-0058-44af-b530-5e68a054d946" volumeName="kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-serving-certs-ca-bundle" seLinuxMountContext="" Nov 04 03:50:02.044723 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044623 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eff746dd-87c2-47fb-bee2-f6ebb6c11fc6" volumeName="kubernetes.io/empty-dir/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-data-volume" seLinuxMountContext="" Nov 04 03:50:02.044723 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044631 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07" volumeName="kubernetes.io/projected/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-kube-api-access-n49rr" seLinuxMountContext="" Nov 04 03:50:02.044723 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044639 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2b70504f-4169-44b2-8b4e-979a068f070d" volumeName="kubernetes.io/projected/2b70504f-4169-44b2-8b4e-979a068f070d-kube-api-access-txw7k" seLinuxMountContext="" Nov 04 03:50:02.044723 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044648 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42510c3d-0283-4901-9d23-a84793598ac9" volumeName="kubernetes.io/secret/42510c3d-0283-4901-9d23-a84793598ac9-ovn-node-metrics-cert" seLinuxMountContext="" Nov 04 03:50:02.044723 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044656 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="56348938-ef44-41ab-8a71-acc1e4308262" volumeName="kubernetes.io/projected/56348938-ef44-41ab-8a71-acc1e4308262-kube-api-access-94ws5" seLinuxMountContext="" Nov 04 03:50:02.044723 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044664 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b1091f60-668b-4884-8548-fe4f4c438367" volumeName="kubernetes.io/secret/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-kube-rbac-proxy-config" seLinuxMountContext="" Nov 04 03:50:02.044723 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044672 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e191ac4e-0058-44af-b530-5e68a054d946" volumeName="kubernetes.io/projected/e191ac4e-0058-44af-b530-5e68a054d946-tls-assets" seLinuxMountContext="" Nov 04 03:50:02.044723 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044679 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eff746dd-87c2-47fb-bee2-f6ebb6c11fc6" volumeName="kubernetes.io/configmap/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-kube-rbac-proxy-cm" seLinuxMountContext="" Nov 04 03:50:02.044723 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044686 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e191ac4e-0058-44af-b530-5e68a054d946" volumeName="kubernetes.io/empty-dir/e191ac4e-0058-44af-b530-5e68a054d946-config-out" seLinuxMountContext="" Nov 04 03:50:02.044723 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044693 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42510c3d-0283-4901-9d23-a84793598ac9" volumeName="kubernetes.io/projected/42510c3d-0283-4901-9d23-a84793598ac9-kube-api-access-9vpwl" seLinuxMountContext="" Nov 04 03:50:02.044723 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044703 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5cf7e388-1678-4afb-8877-30179162401b" volumeName="kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv" seLinuxMountContext="" Nov 04 03:50:02.044723 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044732 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eaec1015-2b2b-4f06-b41d-028671919cf0" volumeName="kubernetes.io/projected/eaec1015-2b2b-4f06-b41d-028671919cf0-kube-api-access-ftm9l" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044740 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eff746dd-87c2-47fb-bee2-f6ebb6c11fc6" volumeName="kubernetes.io/projected/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-kube-api-access-z7hkr" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044749 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e191ac4e-0058-44af-b530-5e68a054d946" volumeName="kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-1" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044779 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06a274ba-93f4-4dbc-ac39-4039fd254e7d" volumeName="kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044790 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2502af7a-1118-4c5e-84b8-8c97188c09a4" volumeName="kubernetes.io/secret/2502af7a-1118-4c5e-84b8-8c97188c09a4-prometheus-operator-kube-rbac-proxy-config" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044801 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="27390c92-0480-4b55-ba3d-e20f7ada5b83" volumeName="kubernetes.io/secret/27390c92-0480-4b55-ba3d-e20f7ada5b83-monitoring-plugin-cert" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044815 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3e07cc7a-88c6-4e29-b7ab-c078916cd977" volumeName="kubernetes.io/empty-dir/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-tuned" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044827 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a6918a9-3ff2-46aa-bc91-affd1835403d" volumeName="kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044841 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2502af7a-1118-4c5e-84b8-8c97188c09a4" volumeName="kubernetes.io/configmap/2502af7a-1118-4c5e-84b8-8c97188c09a4-metrics-client-ca" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044851 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="48e3e3bd-2ec1-4587-97b3-7b37fc5130fb" volumeName="kubernetes.io/projected/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-kube-api-access-4zpwv" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044862 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a6918a9-3ff2-46aa-bc91-affd1835403d" volumeName="kubernetes.io/projected/8a6918a9-3ff2-46aa-bc91-affd1835403d-kube-api-access-hjqhd" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044875 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b1091f60-668b-4884-8548-fe4f4c438367" volumeName="kubernetes.io/projected/b1091f60-668b-4884-8548-fe4f4c438367-kube-api-access-bgmb8" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044886 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8" volumeName="kubernetes.io/configmap/e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8-serviceca" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044904 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e191ac4e-0058-44af-b530-5e68a054d946" volumeName="kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-trusted-ca-bundle" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044916 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e191ac4e-0058-44af-b530-5e68a054d946" volumeName="kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-grpc-tls" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044929 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e191ac4e-0058-44af-b530-5e68a054d946" volumeName="kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-thanos-prometheus-http-client-file" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044941 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07" volumeName="kubernetes.io/configmap/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-cni-binary-copy" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044967 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="48e3e3bd-2ec1-4587-97b3-7b37fc5130fb" volumeName="kubernetes.io/configmap/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-whereabouts-flatfile-configmap" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044981 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="56348938-ef44-41ab-8a71-acc1e4308262" volumeName="kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.044994 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e191ac4e-0058-44af-b530-5e68a054d946" volumeName="kubernetes.io/projected/e191ac4e-0058-44af-b530-5e68a054d946-kube-api-access-zlw9q" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045006 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e191ac4e-0058-44af-b530-5e68a054d946" volumeName="kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-thanos-sidecar-tls" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045017 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e191ac4e-0058-44af-b530-5e68a054d946" volumeName="kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-web-config" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045028 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eff746dd-87c2-47fb-bee2-f6ebb6c11fc6" volumeName="kubernetes.io/secret/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-insights-runtime-extractor-tls" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045039 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2502af7a-1118-4c5e-84b8-8c97188c09a4" volumeName="kubernetes.io/secret/2502af7a-1118-4c5e-84b8-8c97188c09a4-prometheus-operator-tls" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045052 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5bdb7796-b1a3-49a5-81cd-c8101320e172" volumeName="kubernetes.io/configmap/5bdb7796-b1a3-49a5-81cd-c8101320e172-iptables-alerter-script" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045065 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94c06e7a-0510-4787-8b60-45b185300091" volumeName="kubernetes.io/projected/94c06e7a-0510-4787-8b60-45b185300091-kube-api-access-62d4x" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045076 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94c06e7a-0510-4787-8b60-45b185300091" volumeName="kubernetes.io/configmap/94c06e7a-0510-4787-8b60-45b185300091-config-volume" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045099 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b1091f60-668b-4884-8548-fe4f4c438367" volumeName="kubernetes.io/empty-dir/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-textfile" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045110 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b1091f60-668b-4884-8548-fe4f4c438367" volumeName="kubernetes.io/secret/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-tls" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045121 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e191ac4e-0058-44af-b530-5e68a054d946" volumeName="kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-kubelet-serving-ca-bundle" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045134 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e191ac4e-0058-44af-b530-5e68a054d946" volumeName="kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-kube-rbac-proxy-web" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045146 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="27390c92-0480-4b55-ba3d-e20f7ada5b83" volumeName="kubernetes.io/projected/27390c92-0480-4b55-ba3d-e20f7ada5b83-kube-api-access-gf87g" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045159 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2b70504f-4169-44b2-8b4e-979a068f070d" volumeName="kubernetes.io/empty-dir/2b70504f-4169-44b2-8b4e-979a068f070d-tmp-dir" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045170 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="48e3e3bd-2ec1-4587-97b3-7b37fc5130fb" volumeName="kubernetes.io/configmap/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-cni-binary-copy" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045184 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a6918a9-3ff2-46aa-bc91-affd1835403d" volumeName="kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045199 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dde9ca65-5089-450e-97cd-89bf66adba30" volumeName="kubernetes.io/configmap/dde9ca65-5089-450e-97cd-89bf66adba30-konnectivity-ca" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045212 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8" volumeName="kubernetes.io/projected/e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8-kube-api-access-gkcvz" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045224 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e191ac4e-0058-44af-b530-5e68a054d946" volumeName="kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-0" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045238 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42510c3d-0283-4901-9d23-a84793598ac9" volumeName="kubernetes.io/configmap/42510c3d-0283-4901-9d23-a84793598ac9-ovnkube-script-lib" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045250 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5bdb7796-b1a3-49a5-81cd-c8101320e172" volumeName="kubernetes.io/projected/5bdb7796-b1a3-49a5-81cd-c8101320e172-kube-api-access-68d54" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045262 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a6918a9-3ff2-46aa-bc91-affd1835403d" volumeName="kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045297 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a6918a9-3ff2-46aa-bc91-affd1835403d" volumeName="kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-grpc-tls" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045311 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e191ac4e-0058-44af-b530-5e68a054d946" volumeName="kubernetes.io/empty-dir/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-db" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045325 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2502af7a-1118-4c5e-84b8-8c97188c09a4" volumeName="kubernetes.io/projected/2502af7a-1118-4c5e-84b8-8c97188c09a4-kube-api-access-bv8hw" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045339 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="48e3e3bd-2ec1-4587-97b3-7b37fc5130fb" volumeName="kubernetes.io/configmap/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-cni-sysctl-allowlist" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045350 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a6918a9-3ff2-46aa-bc91-affd1835403d" volumeName="kubernetes.io/empty-dir/8a6918a9-3ff2-46aa-bc91-affd1835403d-tmp" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045362 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94c06e7a-0510-4787-8b60-45b185300091" volumeName="kubernetes.io/empty-dir/94c06e7a-0510-4787-8b60-45b185300091-tmp-dir" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045375 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b1091f60-668b-4884-8548-fe4f4c438367" volumeName="kubernetes.io/configmap/b1091f60-668b-4884-8548-fe4f4c438367-metrics-client-ca" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045387 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07" volumeName="kubernetes.io/configmap/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-multus-daemon-config" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045398 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42510c3d-0283-4901-9d23-a84793598ac9" volumeName="kubernetes.io/configmap/42510c3d-0283-4901-9d23-a84793598ac9-ovnkube-config" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045410 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a6918a9-3ff2-46aa-bc91-affd1835403d" volumeName="kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-tls" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045423 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e191ac4e-0058-44af-b530-5e68a054d946" volumeName="kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-kube-rbac-proxy" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045434 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3e07cc7a-88c6-4e29-b7ab-c078916cd977" volumeName="kubernetes.io/empty-dir/3e07cc7a-88c6-4e29-b7ab-c078916cd977-tmp" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045446 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dde9ca65-5089-450e-97cd-89bf66adba30" volumeName="kubernetes.io/secret/dde9ca65-5089-450e-97cd-89bf66adba30-agent-certs" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045459 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e191ac4e-0058-44af-b530-5e68a054d946" volumeName="kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-tls" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045470 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ea2d0497-b79b-4640-b8ca-bd829b1f3dfa" volumeName="kubernetes.io/projected/ea2d0497-b79b-4640-b8ca-bd829b1f3dfa-kube-api-access-jm29k" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045483 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a6918a9-3ff2-46aa-bc91-affd1835403d" volumeName="kubernetes.io/configmap/8a6918a9-3ff2-46aa-bc91-affd1835403d-metrics-client-ca" seLinuxMountContext="" Nov 04 03:50:02.045450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045495 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a6918a9-3ff2-46aa-bc91-affd1835403d" volumeName="kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext="" Nov 04 03:50:02.047929 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045507 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b1091f60-668b-4884-8548-fe4f4c438367" volumeName="kubernetes.io/configmap/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-accelerators-collector-config" seLinuxMountContext="" Nov 04 03:50:02.047929 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045518 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e191ac4e-0058-44af-b530-5e68a054d946" volumeName="kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-metrics-client-certs" seLinuxMountContext="" Nov 04 03:50:02.047929 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045530 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ea2d0497-b79b-4640-b8ca-bd829b1f3dfa" volumeName="kubernetes.io/secret/ea2d0497-b79b-4640-b8ca-bd829b1f3dfa-cert" seLinuxMountContext="" Nov 04 03:50:02.047929 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045541 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3e07cc7a-88c6-4e29-b7ab-c078916cd977" volumeName="kubernetes.io/projected/3e07cc7a-88c6-4e29-b7ab-c078916cd977-kube-api-access-4b9rr" seLinuxMountContext="" Nov 04 03:50:02.047929 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045553 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42510c3d-0283-4901-9d23-a84793598ac9" volumeName="kubernetes.io/configmap/42510c3d-0283-4901-9d23-a84793598ac9-env-overrides" seLinuxMountContext="" Nov 04 03:50:02.047929 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045566 6000 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94c06e7a-0510-4787-8b60-45b185300091" volumeName="kubernetes.io/secret/94c06e7a-0510-4787-8b60-45b185300091-metrics-tls" seLinuxMountContext="" Nov 04 03:50:02.047929 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045578 6000 reconstruct.go:97] "Volume reconstruction finished" Nov 04 03:50:02.047929 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.045586 6000 reconciler.go:29] "Reconciler: start to sync state" Nov 04 03:50:02.047929 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.046150 6000 factory.go:153] Registering CRI-O factory Nov 04 03:50:02.047929 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.046167 6000 factory.go:223] Registration of the crio container factory successfully Nov 04 03:50:02.047929 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.046235 6000 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Nov 04 03:50:02.047929 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.046247 6000 factory.go:55] Registering systemd factory Nov 04 03:50:02.047929 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.046256 6000 factory.go:223] Registration of the systemd container factory successfully Nov 04 03:50:02.047929 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.046273 6000 factory.go:103] Registering Raw factory Nov 04 03:50:02.047929 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.046287 6000 manager.go:1199] Started watching for new ooms in manager Nov 04 03:50:02.047929 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.046658 6000 manager.go:319] Starting recovery of all containers Nov 04 03:50:02.058557 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.058515 6000 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Nov 04 03:50:02.059839 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.059811 6000 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 03:50:02.060522 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.060315 6000 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Nov 04 03:50:02.062567 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.062547 6000 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 03:50:02.064519 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.064498 6000 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 03:50:02.074123 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.074091 6000 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Nov 04 03:50:02.074123 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.074117 6000 status_manager.go:249] "Starting to sync pod status with apiserver" Nov 04 03:50:02.074372 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.074138 6000 kubelet.go:2521] "Starting kubelet main sync loop" Nov 04 03:50:02.074372 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:50:02.074183 6000 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Nov 04 03:50:02.077922 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.077898 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zfgzh_42510c3d-0283-4901-9d23-a84793598ac9/ovn-acl-logging/0.log" Nov 04 03:50:02.078209 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.078193 6000 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 03:50:02.078912 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.078887 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zfgzh_42510c3d-0283-4901-9d23-a84793598ac9/ovn-acl-logging/0.log" Nov 04 03:50:02.086359 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.086039 6000 generic.go:356] "Generic (PLEG): container finished" podID="e191ac4e-0058-44af-b530-5e68a054d946" containerID="30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad" exitCode=0 Nov 04 03:50:02.097004 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.096974 6000 generic.go:356] "Generic (PLEG): container finished" podID="b1091f60-668b-4884-8548-fe4f4c438367" containerID="96557d6e4cf1cfe4b85971ecdcff920b326b464f7cf8f30a4edb83b2b1d338e1" exitCode=0 Nov 04 03:50:02.103031 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.103008 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zfgzh_42510c3d-0283-4901-9d23-a84793598ac9/ovn-acl-logging/0.log" Nov 04 03:50:02.103386 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.103363 6000 generic.go:356] "Generic (PLEG): container finished" podID="42510c3d-0283-4901-9d23-a84793598ac9" containerID="0ada75f8142f065cef35019bcbcb2dee6a63cfe3a6e816dbec9405235c8064fd" exitCode=1 Nov 04 03:50:02.127161 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.127118 6000 generic.go:356] "Generic (PLEG): container finished" podID="295feeff6b2d9f63532642beaf61c650" containerID="231c4c085ba1655bdcc09e68b876000b1bf12834e03b7af23c19c813234e53a3" exitCode=0 Nov 04 03:50:02.132877 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.132829 6000 generic.go:356] "Generic (PLEG): container finished" podID="48e3e3bd-2ec1-4587-97b3-7b37fc5130fb" containerID="552419065ac4d7a094ea549294c5de9824019129aaa0ef78ac28931a84ceed4f" exitCode=0 Nov 04 03:50:02.132877 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.132873 6000 generic.go:356] "Generic (PLEG): container finished" podID="48e3e3bd-2ec1-4587-97b3-7b37fc5130fb" containerID="d3b5e94dafd63e4636c937a9a57887768ec5755e659e01828029c0282bd3fbdc" exitCode=0 Nov 04 03:50:02.133113 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.132885 6000 generic.go:356] "Generic (PLEG): container finished" podID="48e3e3bd-2ec1-4587-97b3-7b37fc5130fb" containerID="aaa944c2a73d5beb680511d9984b12ee93bee1d618ac914fd12f9f2af8c96d7f" exitCode=0 Nov 04 03:50:02.133113 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.132898 6000 generic.go:356] "Generic (PLEG): container finished" podID="48e3e3bd-2ec1-4587-97b3-7b37fc5130fb" containerID="4a9c1ea6b00073ed4fc47deab87fc95db9f2f96d4b41a97debb593bb20942249" exitCode=0 Nov 04 03:50:02.133113 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.132908 6000 generic.go:356] "Generic (PLEG): container finished" podID="48e3e3bd-2ec1-4587-97b3-7b37fc5130fb" containerID="12f37ca2f4f9e5f6f03dad61b237ada4791fb6f394515af5a20d5dee40f980c4" exitCode=0 Nov 04 03:50:02.133113 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.132916 6000 generic.go:356] "Generic (PLEG): container finished" podID="48e3e3bd-2ec1-4587-97b3-7b37fc5130fb" containerID="57f4fbd8debd9105bb20a0ff76091118622217dd8f52b734e646facdb49b3665" exitCode=0 Nov 04 03:50:02.152483 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.152414 6000 manager.go:324] Recovery completed Nov 04 03:50:02.175325 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:50:02.175274 6000 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Nov 04 03:50:02.176618 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.176599 6000 cpu_manager.go:226] "Starting" policy="none" Nov 04 03:50:02.176618 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.176617 6000 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Nov 04 03:50:02.176690 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.176639 6000 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Nov 04 03:50:02.176801 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.176787 6000 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Nov 04 03:50:02.176832 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.176801 6000 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Nov 04 03:50:02.176832 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.176820 6000 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Nov 04 03:50:02.176832 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.176825 6000 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Nov 04 03:50:02.176832 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.176831 6000 policy_none.go:50] "Start" Nov 04 03:50:02.176942 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.176845 6000 memory_manager.go:187] "Starting memorymanager" policy="None" Nov 04 03:50:02.176942 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.176854 6000 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Nov 04 03:50:02.177814 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.177801 6000 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Nov 04 03:50:02.177849 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.177815 6000 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Nov 04 03:50:02.177849 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.177821 6000 policy_none.go:44] "Start" Nov 04 03:50:02.182462 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.182425 6000 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Nov 04 03:50:02.185145 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.185130 6000 manager.go:341] "Starting Device Plugin manager" Nov 04 03:50:02.185898 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:50:02.185159 6000 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Nov 04 03:50:02.186039 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.185904 6000 server.go:87] "Starting device plugin registration server" Nov 04 03:50:02.186279 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.186265 6000 eviction_manager.go:194] "Eviction manager: starting control loop" Nov 04 03:50:02.186309 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.186281 6000 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Nov 04 03:50:02.186396 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.186373 6000 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Nov 04 03:50:02.186520 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.186485 6000 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Nov 04 03:50:02.186575 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.186538 6000 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Nov 04 03:50:02.186575 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.186546 6000 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Nov 04 03:50:02.186668 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.186590 6000 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T03:50:02.186519569Z","UUID":"da501709-1d3b-4999-90af-b8d36df3490a","Handler":null,"Name":"","Endpoint":""} Nov 04 03:50:02.187886 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.187865 6000 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Nov 04 03:50:02.187886 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.187894 6000 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Nov 04 03:50:02.188354 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:50:02.188333 6000 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Nov 04 03:50:02.286912 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.286878 6000 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 03:50:02.289123 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.289100 6000 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-134.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 03:50:02.289256 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.289135 6000 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-134.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 03:50:02.289256 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.289150 6000 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-134.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 04 03:50:02.289256 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.289203 6000 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:50:02.302365 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.302335 6000 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:50:02.302503 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.302425 6000 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:50:02.375932 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.375872 6000 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal"] Nov 04 03:50:02.376615 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.376566 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal" event={"ID":"18d40cab8244943c9879207cd43e3bc3","Type":"ContainerStarted","Data":"b7436ce2746954cf8f807da767a632dbde5647cd9e00233f79800b9efa5ccff3"} Nov 04 03:50:02.376615 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.376614 6000 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:50:02.376814 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.376625 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal" event={"ID":"18d40cab8244943c9879207cd43e3bc3","Type":"ContainerStarted","Data":"1ab58e8cd93b2dc15d1ce30204828c3d59ecad172948a145590ac93dec61b14d"} Nov 04 03:50:02.376814 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.376590 6000 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:50:02.376814 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.376701 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" event={"ID":"295feeff6b2d9f63532642beaf61c650","Type":"ContainerStarted","Data":"7d6b6808d6af8b504747ae2de12f868ec02b2799db72a975ef3b175911598126"} Nov 04 03:50:02.376814 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.376715 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" event={"ID":"295feeff6b2d9f63532642beaf61c650","Type":"ContainerDied","Data":"231c4c085ba1655bdcc09e68b876000b1bf12834e03b7af23c19c813234e53a3"} Nov 04 03:50:02.376814 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.376727 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" event={"ID":"295feeff6b2d9f63532642beaf61c650","Type":"ContainerStarted","Data":"7ead774dcedf1f778aa4931d15c7a1dd22b1748a24133f95dac09e4a3f393764"} Nov 04 03:50:02.391608 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.391580 6000 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 03:50:02.391728 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:50:02.391637 6000 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:50:02.392290 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.392273 6000 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 03:50:02.392328 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:50:02.392315 6000 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:50:02.461719 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.461626 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/295feeff6b2d9f63532642beaf61c650-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal\" (UID: \"295feeff6b2d9f63532642beaf61c650\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:50:02.461719 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.461663 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/18d40cab8244943c9879207cd43e3bc3-config\") pod \"kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal\" (UID: \"18d40cab8244943c9879207cd43e3bc3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:50:02.461719 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.461684 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/295feeff6b2d9f63532642beaf61c650-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal\" (UID: \"295feeff6b2d9f63532642beaf61c650\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:50:02.562328 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.562291 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/295feeff6b2d9f63532642beaf61c650-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal\" (UID: \"295feeff6b2d9f63532642beaf61c650\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:50:02.562328 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.562327 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/295feeff6b2d9f63532642beaf61c650-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal\" (UID: \"295feeff6b2d9f63532642beaf61c650\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:50:02.562528 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.562349 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/18d40cab8244943c9879207cd43e3bc3-config\") pod \"kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal\" (UID: \"18d40cab8244943c9879207cd43e3bc3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:50:02.562528 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.562400 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/18d40cab8244943c9879207cd43e3bc3-config\") pod \"kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal\" (UID: \"18d40cab8244943c9879207cd43e3bc3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:50:02.562528 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.562412 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/295feeff6b2d9f63532642beaf61c650-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal\" (UID: \"295feeff6b2d9f63532642beaf61c650\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:50:02.562528 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:02.562398 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/295feeff6b2d9f63532642beaf61c650-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal\" (UID: \"295feeff6b2d9f63532642beaf61c650\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:50:03.028082 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.028041 6000 apiserver.go:52] "Watching apiserver" Nov 04 03:50:03.045964 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.045929 6000 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Nov 04 03:50:03.046665 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.046636 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-zgj8w","openshift-dns/dns-default-vcxpf","openshift-ingress-canary/ingress-canary-c425l","kube-system/konnectivity-agent-895vv","openshift-monitoring/node-exporter-r7srf","openshift-insights/insights-runtime-extractor-rx686","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59","openshift-multus/multus-additional-cni-plugins-lrdgr","openshift-multus/network-metrics-daemon-p422k","openshift-network-diagnostics/network-check-target-967tl","kube-system/global-pull-secret-syncer-p84rx","kube-system/kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal","openshift-image-registry/node-ca-7vkwm","openshift-monitoring/monitoring-plugin-77f4f76594-f92gv","openshift-monitoring/thanos-querier-7976867cc6-jgp9n","openshift-ovn-kubernetes/ovnkube-node-zfgzh","openshift-dns/node-resolver-dxmg7","openshift-network-operator/iptables-alerter-mm4gp","openshift-monitoring/prometheus-k8s-0","openshift-multus/multus-5kmqc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal","openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p"] Nov 04 03:50:03.052393 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.052358 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Nov 04 03:50:03.054875 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.054842 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-78gsl\"" Nov 04 03:50:03.055141 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.055123 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Nov 04 03:50:03.055242 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.055229 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Nov 04 03:50:03.055409 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.054861 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Nov 04 03:50:03.055669 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.055650 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-x8rgd\"" Nov 04 03:50:03.055783 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.055669 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Nov 04 03:50:03.055783 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.055706 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Nov 04 03:50:03.055898 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.055748 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Nov 04 03:50:03.055898 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.055885 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Nov 04 03:50:03.056017 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.055993 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Nov 04 03:50:03.056093 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.056052 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Nov 04 03:50:03.056093 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.056000 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Nov 04 03:50:03.056193 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.056111 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Nov 04 03:50:03.056193 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.056127 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-j5cz2\"" Nov 04 03:50:03.056474 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.056451 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-qdtd9\"" Nov 04 03:50:03.057648 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.057623 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-p2j7s\"" Nov 04 03:50:03.057744 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.057651 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Nov 04 03:50:03.057744 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.057713 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Nov 04 03:50:03.058132 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.058107 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Nov 04 03:50:03.058230 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.058180 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 04 03:50:03.058283 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.058229 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Nov 04 03:50:03.058397 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.058380 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Nov 04 03:50:03.058450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.058400 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Nov 04 03:50:03.058450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.058435 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Nov 04 03:50:03.058603 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.058450 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4ttlw\"" Nov 04 03:50:03.058603 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.058496 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Nov 04 03:50:03.058603 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.058528 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-m5nwx\"" Nov 04 03:50:03.058776 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.058746 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-twz4j\"" Nov 04 03:50:03.058842 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.058781 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Nov 04 03:50:03.059131 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.059111 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fmfbg\"" Nov 04 03:50:03.059205 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.059145 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Nov 04 03:50:03.059205 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.059167 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Nov 04 03:50:03.059205 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.059179 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Nov 04 03:50:03.059205 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.059195 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tl6ng\"" Nov 04 03:50:03.059205 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.059204 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Nov 04 03:50:03.059505 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.059487 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Nov 04 03:50:03.059920 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.059901 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-75zvg\"" Nov 04 03:50:03.059920 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.059902 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 04 03:50:03.060049 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.059976 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-49g8t\"" Nov 04 03:50:03.060049 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.060005 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Nov 04 03:50:03.060149 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.060051 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 04 03:50:03.060149 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.060137 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 04 03:50:03.060224 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.060158 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Nov 04 03:50:03.060260 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.060235 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ccqtd\"" Nov 04 03:50:03.060309 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.060292 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Nov 04 03:50:03.060959 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.060939 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Nov 04 03:50:03.061040 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.060961 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Nov 04 03:50:03.061040 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.060986 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Nov 04 03:50:03.061112 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.061035 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Nov 04 03:50:03.061837 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.061816 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Nov 04 03:50:03.061837 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.061836 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Nov 04 03:50:03.061998 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.061817 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Nov 04 03:50:03.061998 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.061907 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-n475p\"" Nov 04 03:50:03.061998 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.061921 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Nov 04 03:50:03.061998 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.061994 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-49k7r\"" Nov 04 03:50:03.062503 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.062488 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Nov 04 03:50:03.062557 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.062546 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Nov 04 03:50:03.062648 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.062629 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Nov 04 03:50:03.062737 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.062693 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Nov 04 03:50:03.062803 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.062745 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Nov 04 03:50:03.062836 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.062825 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rh4kv\"" Nov 04 03:50:03.062881 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.062868 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Nov 04 03:50:03.062923 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.062912 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-s9kfd\"" Nov 04 03:50:03.063734 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.063719 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Nov 04 03:50:03.063858 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.063845 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-7cu829nqfpjf\"" Nov 04 03:50:03.064351 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.064332 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Nov 04 03:50:03.064409 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.064333 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Nov 04 03:50:03.064409 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.064354 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-qc8ts\"" Nov 04 03:50:03.064409 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.064355 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Nov 04 03:50:03.064409 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.064399 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Nov 04 03:50:03.064648 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.064627 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-d55th\"" Nov 04 03:50:03.064719 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.064701 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Nov 04 03:50:03.064799 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.064784 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Nov 04 03:50:03.064984 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.064970 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Nov 04 03:50:03.066044 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.066024 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Nov 04 03:50:03.066126 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.066065 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Nov 04 03:50:03.066126 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.066108 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Nov 04 03:50:03.066207 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.066186 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Nov 04 03:50:03.066331 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.066230 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Nov 04 03:50:03.066331 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.066235 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-bjr97m2gdpk1p\"" Nov 04 03:50:03.066331 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.066234 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Nov 04 03:50:03.066331 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.066238 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Nov 04 03:50:03.066331 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.066265 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Nov 04 03:50:03.066331 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.066329 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Nov 04 03:50:03.066544 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.066363 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Nov 04 03:50:03.066544 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.066396 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Nov 04 03:50:03.067235 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.067220 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Nov 04 03:50:03.067309 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.067292 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bsczw\"" Nov 04 03:50:03.075030 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.075005 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Nov 04 03:50:03.075137 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.075123 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Nov 04 03:50:03.136578 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.136545 6000 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:50:03.136744 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.136641 6000 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:50:03.143891 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.143864 6000 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Nov 04 03:50:03.148870 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.148835 6000 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 03:50:03.148975 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:50:03.148900 6000 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:50:03.149800 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.149779 6000 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 03:50:03.149901 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:50:03.149832 6000 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-134.us-east-2.compute.internal" Nov 04 03:50:03.164953 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.164922 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8-serviceca\") pod \"node-ca-7vkwm\" (UID: \"e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8\") " pod="openshift-image-registry/node-ca-7vkwm" Nov 04 03:50:03.165093 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.164957 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.165093 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.164983 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-var-lib-openvswitch\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.165093 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.165036 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vpwl\" (UniqueName: \"kubernetes.io/projected/42510c3d-0283-4901-9d23-a84793598ac9-kube-api-access-9vpwl\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.165211 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.165195 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv8hw\" (UniqueName: \"kubernetes.io/projected/2502af7a-1118-4c5e-84b8-8c97188c09a4-kube-api-access-bv8hw\") pod \"prometheus-operator-9cd89b74c-tnr8p\" (UID: \"2502af7a-1118-4c5e-84b8-8c97188c09a4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" Nov 04 03:50:03.165243 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.165223 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:50:03.165287 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.165243 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-registration-dir\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:50:03.165287 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.165267 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-var-lib-cni-bin\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.165384 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.165290 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-multus-daemon-config\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.165384 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.165317 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea2d0497-b79b-4640-b8ca-bd829b1f3dfa-cert\") pod \"ingress-canary-c425l\" (UID: \"ea2d0497-b79b-4640-b8ca-bd829b1f3dfa\") " pod="openshift-ingress-canary/ingress-canary-c425l" Nov 04 03:50:03.165485 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.165430 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.165554 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.165497 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:50:03.165554 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.165524 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-grpc-tls\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:50:03.165655 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.165553 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.165655 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.165579 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlw9q\" (UniqueName: \"kubernetes.io/projected/e191ac4e-0058-44af-b530-5e68a054d946-kube-api-access-zlw9q\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.165785 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.165652 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-system-cni-dir\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:50:03.165785 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.165658 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:50:03.165785 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.165673 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.165785 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.165680 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-var-lib-cni-multus\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.165785 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.165607 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea2d0497-b79b-4640-b8ca-bd829b1f3dfa-cert\") pod \"ingress-canary-c425l\" (UID: \"ea2d0497-b79b-4640-b8ca-bd829b1f3dfa\") " pod="openshift-ingress-canary/ingress-canary-c425l" Nov 04 03:50:03.165785 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.165708 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8-serviceca\") pod \"node-ca-7vkwm\" (UID: \"e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8\") " pod="openshift-image-registry/node-ca-7vkwm" Nov 04 03:50:03.165785 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.165739 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-multus-daemon-config\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.166243 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.165893 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-grpc-tls\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:50:03.166243 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166051 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.166243 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166134 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:50:03.166243 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.165876 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-hostroot\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.166243 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166213 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.166243 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166241 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94c06e7a-0510-4787-8b60-45b185300091-tmp-dir\") pod \"dns-default-vcxpf\" (UID: \"94c06e7a-0510-4787-8b60-45b185300091\") " pod="openshift-dns/dns-default-vcxpf" Nov 04 03:50:03.166464 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166267 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-device-dir\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:50:03.166464 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166292 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/06a274ba-93f4-4dbc-ac39-4039fd254e7d-kubelet-config\") pod \"global-pull-secret-syncer-p84rx\" (UID: \"06a274ba-93f4-4dbc-ac39-4039fd254e7d\") " pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:50:03.166464 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166316 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-crio-socket\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:50:03.166464 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166361 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.166464 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166384 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-lib-modules\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.166464 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166407 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-host\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.166464 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166432 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-run-netns\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.166890 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166461 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.166890 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166495 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2502af7a-1118-4c5e-84b8-8c97188c09a4-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-tnr8p\" (UID: \"2502af7a-1118-4c5e-84b8-8c97188c09a4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" Nov 04 03:50:03.166890 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166503 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.166890 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166520 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a6918a9-3ff2-46aa-bc91-affd1835403d-metrics-client-ca\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:50:03.166890 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166546 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.166890 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166559 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.166890 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166592 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/42510c3d-0283-4901-9d23-a84793598ac9-ovn-node-metrics-cert\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.166890 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166613 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-var-lib-kubelet\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.166890 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166674 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2502af7a-1118-4c5e-84b8-8c97188c09a4-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-tnr8p\" (UID: \"2502af7a-1118-4c5e-84b8-8c97188c09a4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" Nov 04 03:50:03.166890 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166703 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1091f60-668b-4884-8548-fe4f4c438367-metrics-client-ca\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:50:03.166890 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166719 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.166890 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166777 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txw7k\" (UniqueName: \"kubernetes.io/projected/2b70504f-4169-44b2-8b4e-979a068f070d-kube-api-access-txw7k\") pod \"node-resolver-dxmg7\" (UID: \"2b70504f-4169-44b2-8b4e-979a068f070d\") " pod="openshift-dns/node-resolver-dxmg7" Nov 04 03:50:03.166890 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166891 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2b70504f-4169-44b2-8b4e-979a068f070d-hosts-file\") pod \"node-resolver-dxmg7\" (UID: \"2b70504f-4169-44b2-8b4e-979a068f070d\") " pod="openshift-dns/node-resolver-dxmg7" Nov 04 03:50:03.167631 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166920 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2502af7a-1118-4c5e-84b8-8c97188c09a4-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-tnr8p\" (UID: \"2502af7a-1118-4c5e-84b8-8c97188c09a4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" Nov 04 03:50:03.167631 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166928 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/42510c3d-0283-4901-9d23-a84793598ac9-ovn-node-metrics-cert\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.167631 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.166960 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94c06e7a-0510-4787-8b60-45b185300091-tmp-dir\") pod \"dns-default-vcxpf\" (UID: \"94c06e7a-0510-4787-8b60-45b185300091\") " pod="openshift-dns/dns-default-vcxpf" Nov 04 03:50:03.167631 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167029 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1091f60-668b-4884-8548-fe4f4c438367-metrics-client-ca\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:50:03.167631 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167074 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a6918a9-3ff2-46aa-bc91-affd1835403d-metrics-client-ca\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:50:03.167631 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167081 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jm29k\" (UniqueName: \"kubernetes.io/projected/ea2d0497-b79b-4640-b8ca-bd829b1f3dfa-kube-api-access-jm29k\") pod \"ingress-canary-c425l\" (UID: \"ea2d0497-b79b-4640-b8ca-bd829b1f3dfa\") " pod="openshift-ingress-canary/ingress-canary-c425l" Nov 04 03:50:03.167631 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167176 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-slash\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.167631 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167194 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-run-netns\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.167631 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167194 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.167631 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167206 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.167631 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167220 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:50:03.167631 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167247 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2502af7a-1118-4c5e-84b8-8c97188c09a4-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-tnr8p\" (UID: \"2502af7a-1118-4c5e-84b8-8c97188c09a4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" Nov 04 03:50:03.167631 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167286 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-var-lib-kubelet\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.167631 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167320 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:50:03.167631 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167349 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-cni-bin\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.167631 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167375 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/42510c3d-0283-4901-9d23-a84793598ac9-ovnkube-config\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.167631 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167397 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-sys\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.167631 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167398 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:50:03.167631 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167537 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:50:03.167631 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167415 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2502af7a-1118-4c5e-84b8-8c97188c09a4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-tnr8p\" (UID: \"2502af7a-1118-4c5e-84b8-8c97188c09a4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" Nov 04 03:50:03.167631 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167585 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/42510c3d-0283-4901-9d23-a84793598ac9-ovnkube-config\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167666 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lvzv\" (UniqueName: \"kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv\") pod \"network-check-target-967tl\" (UID: \"5cf7e388-1678-4afb-8877-30179162401b\") " pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167691 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-system-cni-dir\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167707 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-multus-cni-dir\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167723 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dde9ca65-5089-450e-97cd-89bf66adba30-konnectivity-ca\") pod \"konnectivity-agent-895vv\" (UID: \"dde9ca65-5089-450e-97cd-89bf66adba30\") " pod="kube-system/konnectivity-agent-895vv" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167750 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-config\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167801 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2502af7a-1118-4c5e-84b8-8c97188c09a4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-tnr8p\" (UID: \"2502af7a-1118-4c5e-84b8-8c97188c09a4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167819 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167853 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a6918a9-3ff2-46aa-bc91-affd1835403d-tmp\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167878 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167887 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjqhd\" (UniqueName: \"kubernetes.io/projected/8a6918a9-3ff2-46aa-bc91-affd1835403d-kube-api-access-hjqhd\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167912 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a6918a9-3ff2-46aa-bc91-affd1835403d-tmp\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167915 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e191ac4e-0058-44af-b530-5e68a054d946-config-out\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167950 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-config\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167947 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-web-config\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167988 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dde9ca65-5089-450e-97cd-89bf66adba30-konnectivity-ca\") pod \"konnectivity-agent-895vv\" (UID: \"dde9ca65-5089-450e-97cd-89bf66adba30\") " pod="kube-system/konnectivity-agent-895vv" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.167993 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/42510c3d-0283-4901-9d23-a84793598ac9-ovnkube-script-lib\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168025 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e191ac4e-0058-44af-b530-5e68a054d946-config-out\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168034 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-kubernetes\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168054 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e07cc7a-88c6-4e29-b7ab-c078916cd977-tmp\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168070 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkcvz\" (UniqueName: \"kubernetes.io/projected/e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8-kube-api-access-gkcvz\") pod \"node-ca-7vkwm\" (UID: \"e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8\") " pod="openshift-image-registry/node-ca-7vkwm" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168102 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-etc-kubernetes\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168122 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168134 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e07cc7a-88c6-4e29-b7ab-c078916cd977-tmp\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168141 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e191ac4e-0058-44af-b530-5e68a054d946-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168158 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/42510c3d-0283-4901-9d23-a84793598ac9-ovnkube-script-lib\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168158 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-cnibin\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168171 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-web-config\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168200 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-sysctl-conf\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168259 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-run\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168302 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-accelerators-collector-config\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168335 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e191ac4e-0058-44af-b530-5e68a054d946-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168342 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-kubelet\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168338 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168366 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-cni-binary-copy\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168404 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-tls\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168435 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-run-multus-certs\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168463 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dde9ca65-5089-450e-97cd-89bf66adba30-agent-certs\") pod \"konnectivity-agent-895vv\" (UID: \"dde9ca65-5089-450e-97cd-89bf66adba30\") " pod="kube-system/konnectivity-agent-895vv" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168486 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-accelerators-collector-config\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168498 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret\") pod \"global-pull-secret-syncer-p84rx\" (UID: \"06a274ba-93f4-4dbc-ac39-4039fd254e7d\") " pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168521 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/27390c92-0480-4b55-ba3d-e20f7ada5b83-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-f92gv\" (UID: \"27390c92-0480-4b55-ba3d-e20f7ada5b83\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-f92gv" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168500 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-cni-binary-copy\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168621 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b1091f60-668b-4884-8548-fe4f4c438367-sys\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:50:03.168593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168625 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-tls\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168661 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dde9ca65-5089-450e-97cd-89bf66adba30-agent-certs\") pod \"konnectivity-agent-895vv\" (UID: \"dde9ca65-5089-450e-97cd-89bf66adba30\") " pod="kube-system/konnectivity-agent-895vv" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168703 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs\") pod \"network-metrics-daemon-p422k\" (UID: \"56348938-ef44-41ab-8a71-acc1e4308262\") " pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168729 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n49rr\" (UniqueName: \"kubernetes.io/projected/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-kube-api-access-n49rr\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168765 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168788 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-tuned\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168822 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168841 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7hkr\" (UniqueName: \"kubernetes.io/projected/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-kube-api-access-z7hkr\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168867 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168836 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/27390c92-0480-4b55-ba3d-e20f7ada5b83-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-f92gv\" (UID: \"27390c92-0480-4b55-ba3d-e20f7ada5b83\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-f92gv" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168902 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168904 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/06a274ba-93f4-4dbc-ac39-4039fd254e7d-original-pull-secret\") pod \"global-pull-secret-syncer-p84rx\" (UID: \"06a274ba-93f4-4dbc-ac39-4039fd254e7d\") " pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168946 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62d4x\" (UniqueName: \"kubernetes.io/projected/94c06e7a-0510-4787-8b60-45b185300091-kube-api-access-62d4x\") pod \"dns-default-vcxpf\" (UID: \"94c06e7a-0510-4787-8b60-45b185300091\") " pod="openshift-dns/dns-default-vcxpf" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168972 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b70504f-4169-44b2-8b4e-979a068f070d-tmp-dir\") pod \"node-resolver-dxmg7\" (UID: \"2b70504f-4169-44b2-8b4e-979a068f070d\") " pod="openshift-dns/node-resolver-dxmg7" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168971 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56348938-ef44-41ab-8a71-acc1e4308262-metrics-certs\") pod \"network-metrics-daemon-p422k\" (UID: \"56348938-ef44-41ab-8a71-acc1e4308262\") " pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.168997 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-sysctl-d\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.169001 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.169023 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5bdb7796-b1a3-49a5-81cd-c8101320e172-host-slash\") pod \"iptables-alerter-mm4gp\" (UID: \"5bdb7796-b1a3-49a5-81cd-c8101320e172\") " pod="openshift-network-operator/iptables-alerter-mm4gp" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.169068 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.169110 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.169117 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.169132 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-tuned\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.169141 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b70504f-4169-44b2-8b4e-979a068f070d-tmp-dir\") pod \"node-resolver-dxmg7\" (UID: \"2b70504f-4169-44b2-8b4e-979a068f070d\") " pod="openshift-dns/node-resolver-dxmg7" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.169170 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-multus-socket-dir-parent\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.169224 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-data-volume\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.169244 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-systemd\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.169279 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68d54\" (UniqueName: \"kubernetes.io/projected/5bdb7796-b1a3-49a5-81cd-c8101320e172-kube-api-access-68d54\") pod \"iptables-alerter-mm4gp\" (UID: \"5bdb7796-b1a3-49a5-81cd-c8101320e172\") " pod="openshift-network-operator/iptables-alerter-mm4gp" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.169310 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-data-volume\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.169322 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-tls\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.169458 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftm9l\" (UniqueName: \"kubernetes.io/projected/eaec1015-2b2b-4f06-b41d-028671919cf0-kube-api-access-ftm9l\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.169492 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8a6918a9-3ff2-46aa-bc91-affd1835403d-secret-thanos-querier-tls\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.169533 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8-host\") pod \"node-ca-7vkwm\" (UID: \"e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8\") " pod="openshift-image-registry/node-ca-7vkwm" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.169648 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-cni-binary-copy\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.169678 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-modprobe-d\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.169797 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-textfile\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.169822 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.169843 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-node-log\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.169885 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-textfile\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.169859 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5bdb7796-b1a3-49a5-81cd-c8101320e172-iptables-alerter-script\") pod \"iptables-alerter-mm4gp\" (UID: \"5bdb7796-b1a3-49a5-81cd-c8101320e172\") " pod="openshift-network-operator/iptables-alerter-mm4gp" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.169915 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-cni-binary-copy\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.169986 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170015 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170031 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170042 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5bdb7796-b1a3-49a5-81cd-c8101320e172-iptables-alerter-script\") pod \"iptables-alerter-mm4gp\" (UID: \"5bdb7796-b1a3-49a5-81cd-c8101320e172\") " pod="openshift-network-operator/iptables-alerter-mm4gp" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170063 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-run-ovn-kubernetes\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170090 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/42510c3d-0283-4901-9d23-a84793598ac9-env-overrides\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170112 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170124 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170154 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b1091f60-668b-4884-8548-fe4f4c438367-root\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170182 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgmb8\" (UniqueName: \"kubernetes.io/projected/b1091f60-668b-4884-8548-fe4f4c438367-kube-api-access-bgmb8\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170218 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94ws5\" (UniqueName: \"kubernetes.io/projected/56348938-ef44-41ab-8a71-acc1e4308262-kube-api-access-94ws5\") pod \"network-metrics-daemon-p422k\" (UID: \"56348938-ef44-41ab-8a71-acc1e4308262\") " pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170238 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-run-k8s-cni-cncf-io\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170236 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/42510c3d-0283-4901-9d23-a84793598ac9-env-overrides\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170254 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170278 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zpwv\" (UniqueName: \"kubernetes.io/projected/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-kube-api-access-4zpwv\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170313 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170350 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-sysconfig\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.170338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170371 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170386 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-socket-dir\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170410 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170432 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-log-socket\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170464 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf87g\" (UniqueName: \"kubernetes.io/projected/27390c92-0480-4b55-ba3d-e20f7ada5b83-kube-api-access-gf87g\") pod \"monitoring-plugin-77f4f76594-f92gv\" (UID: \"27390c92-0480-4b55-ba3d-e20f7ada5b83\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-f92gv" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170551 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170646 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-run-ovn\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170680 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-cni-netd\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170695 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170718 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b9rr\" (UniqueName: \"kubernetes.io/projected/3e07cc7a-88c6-4e29-b7ab-c078916cd977-kube-api-access-4b9rr\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170767 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/94c06e7a-0510-4787-8b60-45b185300091-config-volume\") pod \"dns-default-vcxpf\" (UID: \"94c06e7a-0510-4787-8b60-45b185300091\") " pod="openshift-dns/dns-default-vcxpf" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170808 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-etc-selinux\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170810 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170858 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-sys-fs\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170884 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170919 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-systemd-units\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170945 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/94c06e7a-0510-4787-8b60-45b185300091-metrics-tls\") pod \"dns-default-vcxpf\" (UID: \"94c06e7a-0510-4787-8b60-45b185300091\") " pod="openshift-dns/dns-default-vcxpf" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170961 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-cnibin\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170977 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/06a274ba-93f4-4dbc-ac39-4039fd254e7d-dbus\") pod \"global-pull-secret-syncer-p84rx\" (UID: \"06a274ba-93f4-4dbc-ac39-4039fd254e7d\") " pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.170943 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/94c06e7a-0510-4787-8b60-45b185300091-config-volume\") pod \"dns-default-vcxpf\" (UID: \"94c06e7a-0510-4787-8b60-45b185300091\") " pod="openshift-dns/dns-default-vcxpf" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.171001 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.171038 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-run-systemd\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.171056 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-etc-openvswitch\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.171071 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-multus-conf-dir\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.171090 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-run-openvswitch\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.171116 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/94c06e7a-0510-4787-8b60-45b185300091-metrics-tls\") pod \"dns-default-vcxpf\" (UID: \"94c06e7a-0510-4787-8b60-45b185300091\") " pod="openshift-dns/dns-default-vcxpf" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.171128 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.171147 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-wtmp\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:50:03.172333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.171333 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.175565 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.175546 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vpwl\" (UniqueName: \"kubernetes.io/projected/42510c3d-0283-4901-9d23-a84793598ac9-kube-api-access-9vpwl\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.175619 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.175601 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv8hw\" (UniqueName: \"kubernetes.io/projected/2502af7a-1118-4c5e-84b8-8c97188c09a4-kube-api-access-bv8hw\") pod \"prometheus-operator-9cd89b74c-tnr8p\" (UID: \"2502af7a-1118-4c5e-84b8-8c97188c09a4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-tnr8p" Nov 04 03:50:03.176570 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.176554 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlw9q\" (UniqueName: \"kubernetes.io/projected/e191ac4e-0058-44af-b530-5e68a054d946-kube-api-access-zlw9q\") pod \"prometheus-k8s-0\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:03.177803 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.177787 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txw7k\" (UniqueName: \"kubernetes.io/projected/2b70504f-4169-44b2-8b4e-979a068f070d-kube-api-access-txw7k\") pod \"node-resolver-dxmg7\" (UID: \"2b70504f-4169-44b2-8b4e-979a068f070d\") " pod="openshift-dns/node-resolver-dxmg7" Nov 04 03:50:03.178646 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.178630 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lvzv\" (UniqueName: \"kubernetes.io/projected/5cf7e388-1678-4afb-8877-30179162401b-kube-api-access-8lvzv\") pod \"network-check-target-967tl\" (UID: \"5cf7e388-1678-4afb-8877-30179162401b\") " pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:50:03.179404 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.179388 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm29k\" (UniqueName: \"kubernetes.io/projected/ea2d0497-b79b-4640-b8ca-bd829b1f3dfa-kube-api-access-jm29k\") pod \"ingress-canary-c425l\" (UID: \"ea2d0497-b79b-4640-b8ca-bd829b1f3dfa\") " pod="openshift-ingress-canary/ingress-canary-c425l" Nov 04 03:50:03.185793 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.185723 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7hkr\" (UniqueName: \"kubernetes.io/projected/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-kube-api-access-z7hkr\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:50:03.185793 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.185728 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n49rr\" (UniqueName: \"kubernetes.io/projected/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-kube-api-access-n49rr\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.185957 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.185855 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62d4x\" (UniqueName: \"kubernetes.io/projected/94c06e7a-0510-4787-8b60-45b185300091-kube-api-access-62d4x\") pod \"dns-default-vcxpf\" (UID: \"94c06e7a-0510-4787-8b60-45b185300091\") " pod="openshift-dns/dns-default-vcxpf" Nov 04 03:50:03.185957 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.185899 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkcvz\" (UniqueName: \"kubernetes.io/projected/e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8-kube-api-access-gkcvz\") pod \"node-ca-7vkwm\" (UID: \"e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8\") " pod="openshift-image-registry/node-ca-7vkwm" Nov 04 03:50:03.186169 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.186150 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftm9l\" (UniqueName: \"kubernetes.io/projected/eaec1015-2b2b-4f06-b41d-028671919cf0-kube-api-access-ftm9l\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:50:03.186225 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.186186 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68d54\" (UniqueName: \"kubernetes.io/projected/5bdb7796-b1a3-49a5-81cd-c8101320e172-kube-api-access-68d54\") pod \"iptables-alerter-mm4gp\" (UID: \"5bdb7796-b1a3-49a5-81cd-c8101320e172\") " pod="openshift-network-operator/iptables-alerter-mm4gp" Nov 04 03:50:03.186355 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.186337 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjqhd\" (UniqueName: \"kubernetes.io/projected/8a6918a9-3ff2-46aa-bc91-affd1835403d-kube-api-access-hjqhd\") pod \"thanos-querier-7976867cc6-jgp9n\" (UID: \"8a6918a9-3ff2-46aa-bc91-affd1835403d\") " pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:50:03.190661 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.190640 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zpwv\" (UniqueName: \"kubernetes.io/projected/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-kube-api-access-4zpwv\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:50:03.190737 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.190673 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgmb8\" (UniqueName: \"kubernetes.io/projected/b1091f60-668b-4884-8548-fe4f4c438367-kube-api-access-bgmb8\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:50:03.190837 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.190821 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b9rr\" (UniqueName: \"kubernetes.io/projected/3e07cc7a-88c6-4e29-b7ab-c078916cd977-kube-api-access-4b9rr\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.190918 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.190885 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94ws5\" (UniqueName: \"kubernetes.io/projected/56348938-ef44-41ab-8a71-acc1e4308262-kube-api-access-94ws5\") pod \"network-metrics-daemon-p422k\" (UID: \"56348938-ef44-41ab-8a71-acc1e4308262\") " pod="openshift-multus/network-metrics-daemon-p422k" Nov 04 03:50:03.191089 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.191074 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf87g\" (UniqueName: \"kubernetes.io/projected/27390c92-0480-4b55-ba3d-e20f7ada5b83-kube-api-access-gf87g\") pod \"monitoring-plugin-77f4f76594-f92gv\" (UID: \"27390c92-0480-4b55-ba3d-e20f7ada5b83\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-f92gv" Nov 04 03:50:03.271768 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.271716 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-run-systemd\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.272162 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.271750 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-etc-openvswitch\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.272162 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.271809 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-multus-conf-dir\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.272162 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.271824 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-run-openvswitch\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.272162 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.271840 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-wtmp\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:50:03.272162 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.271840 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-run-systemd\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.272162 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.271895 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-multus-conf-dir\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.272162 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.271914 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-run-openvswitch\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.272162 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.271921 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-etc-openvswitch\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.272162 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.271919 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b1091f60-668b-4884-8548-fe4f4c438367-node-exporter-wtmp\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:50:03.272647 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.272605 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-var-lib-openvswitch\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.273128 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.273050 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-var-lib-openvswitch\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.273128 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.273122 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-registration-dir\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:50:03.273303 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.273175 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-var-lib-cni-bin\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.273303 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.273215 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-system-cni-dir\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:50:03.273303 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.273264 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-var-lib-cni-multus\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.273303 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.273292 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-hostroot\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.273497 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.273339 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-device-dir\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:50:03.273497 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.273370 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/06a274ba-93f4-4dbc-ac39-4039fd254e7d-kubelet-config\") pod \"global-pull-secret-syncer-p84rx\" (UID: \"06a274ba-93f4-4dbc-ac39-4039fd254e7d\") " pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:50:03.273497 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.273465 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-system-cni-dir\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:50:03.273497 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.273479 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/06a274ba-93f4-4dbc-ac39-4039fd254e7d-kubelet-config\") pod \"global-pull-secret-syncer-p84rx\" (UID: \"06a274ba-93f4-4dbc-ac39-4039fd254e7d\") " pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:50:03.273687 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.273561 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-registration-dir\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:50:03.273687 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.273625 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-crio-socket\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:50:03.274830 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.274465 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-lib-modules\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.274830 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.274557 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-host\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.274998 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.274934 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-run-netns\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.275068 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.274999 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-var-lib-kubelet\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.275120 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.275062 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2b70504f-4169-44b2-8b4e-979a068f070d-hosts-file\") pod \"node-resolver-dxmg7\" (UID: \"2b70504f-4169-44b2-8b4e-979a068f070d\") " pod="openshift-dns/node-resolver-dxmg7" Nov 04 03:50:03.275173 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.275115 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-slash\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.275173 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.275143 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-run-netns\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.275258 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.275208 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-var-lib-kubelet\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.275258 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.275241 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-cni-bin\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.275351 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.275284 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-sys\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.275351 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.275316 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-system-cni-dir\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.275448 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.275354 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-multus-cni-dir\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.275448 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.275393 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-kubernetes\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.275448 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.275438 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-etc-kubernetes\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.275591 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.275473 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-cnibin\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:50:03.275591 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.275517 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-sysctl-conf\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.275591 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.275541 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-run\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.275591 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.275583 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-kubelet\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.275804 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.275615 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-run-multus-certs\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.275804 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.275663 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b1091f60-668b-4884-8548-fe4f4c438367-sys\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:50:03.275804 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.275700 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:50:03.275804 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.275737 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-sysctl-d\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.275804 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.275786 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5bdb7796-b1a3-49a5-81cd-c8101320e172-host-slash\") pod \"iptables-alerter-mm4gp\" (UID: \"5bdb7796-b1a3-49a5-81cd-c8101320e172\") " pod="openshift-network-operator/iptables-alerter-mm4gp" Nov 04 03:50:03.276031 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.275818 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:50:03.276031 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.275864 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-multus-socket-dir-parent\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.276031 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.275898 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-systemd\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.276031 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.275953 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8-host\") pod \"node-ca-7vkwm\" (UID: \"e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8\") " pod="openshift-image-registry/node-ca-7vkwm" Nov 04 03:50:03.276031 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.276004 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-modprobe-d\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.276272 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.276050 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-node-log\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.276272 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.276110 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-run-ovn-kubernetes\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.276272 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.276171 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b1091f60-668b-4884-8548-fe4f4c438367-root\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:50:03.276272 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.276208 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-run-k8s-cni-cncf-io\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.276272 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.276251 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.276506 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.276285 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-sysconfig\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.276506 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.276331 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-socket-dir\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:50:03.276506 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.276367 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-log-socket\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.276506 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.276415 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-run-ovn\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.276506 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.276445 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-cni-netd\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.276506 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.276494 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-etc-selinux\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:50:03.276824 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.276526 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-sys-fs\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:50:03.276824 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.276574 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-systemd-units\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.276824 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.276602 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-cnibin\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.276824 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.276646 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/06a274ba-93f4-4dbc-ac39-4039fd254e7d-dbus\") pod \"global-pull-secret-syncer-p84rx\" (UID: \"06a274ba-93f4-4dbc-ac39-4039fd254e7d\") " pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:50:03.277020 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.276864 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/06a274ba-93f4-4dbc-ac39-4039fd254e7d-dbus\") pod \"global-pull-secret-syncer-p84rx\" (UID: \"06a274ba-93f4-4dbc-ac39-4039fd254e7d\") " pod="kube-system/global-pull-secret-syncer-p84rx" Nov 04 03:50:03.277020 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.273654 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-hostroot\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.277117 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.277033 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-lib-modules\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.277117 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.273738 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-var-lib-cni-multus\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.277210 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.277116 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-host\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.277210 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.273787 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-var-lib-cni-bin\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.277210 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.277196 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-run-netns\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.277210 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.273803 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-device-dir\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:50:03.277393 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.277284 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-var-lib-kubelet\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.277393 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.273989 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eff746dd-87c2-47fb-bee2-f6ebb6c11fc6-crio-socket\") pod \"insights-runtime-extractor-rx686\" (UID: \"eff746dd-87c2-47fb-bee2-f6ebb6c11fc6\") " pod="openshift-insights/insights-runtime-extractor-rx686" Nov 04 03:50:03.277393 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.277383 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2b70504f-4169-44b2-8b4e-979a068f070d-hosts-file\") pod \"node-resolver-dxmg7\" (UID: \"2b70504f-4169-44b2-8b4e-979a068f070d\") " pod="openshift-dns/node-resolver-dxmg7" Nov 04 03:50:03.277542 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.277434 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-slash\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.277542 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.277501 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-run-netns\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.277635 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.277563 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-var-lib-kubelet\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.277714 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.277691 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-systemd\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.277808 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.277788 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-sys\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.277894 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.277877 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-system-cni-dir\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.278000 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.277967 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-multus-cni-dir\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.278055 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278042 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-kubernetes\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.278153 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278106 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-etc-kubernetes\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.278217 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.277647 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-cni-bin\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.278217 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278179 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-cnibin\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:50:03.278312 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278236 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-run\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.278379 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278310 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-modprobe-d\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.278379 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278358 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-node-log\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.278379 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278364 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-sysctl-conf\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.278525 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278406 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-run-ovn-kubernetes\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.278525 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278414 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-kubelet\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.278525 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278466 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-run-multus-certs\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.278525 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278477 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.278710 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278557 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-etc-selinux\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:50:03.278710 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278560 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-run-ovn\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.278710 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278569 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8-host\") pod \"node-ca-7vkwm\" (UID: \"e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8\") " pod="openshift-image-registry/node-ca-7vkwm" Nov 04 03:50:03.278710 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278560 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b1091f60-668b-4884-8548-fe4f4c438367-root\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:50:03.278710 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278608 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-multus-socket-dir-parent\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.278710 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278632 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-host-cni-netd\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.278710 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278639 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-sysctl-d\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.278710 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278649 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-socket-dir\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:50:03.278710 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278675 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5bdb7796-b1a3-49a5-81cd-c8101320e172-host-slash\") pod \"iptables-alerter-mm4gp\" (UID: \"5bdb7796-b1a3-49a5-81cd-c8101320e172\") " pod="openshift-network-operator/iptables-alerter-mm4gp" Nov 04 03:50:03.278710 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278701 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-cnibin\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.278710 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278709 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3e07cc7a-88c6-4e29-b7ab-c078916cd977-etc-sysconfig\") pod \"tuned-zgj8w\" (UID: \"3e07cc7a-88c6-4e29-b7ab-c078916cd977\") " pod="openshift-cluster-node-tuning-operator/tuned-zgj8w" Nov 04 03:50:03.278710 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278711 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07-host-run-k8s-cni-cncf-io\") pod \"multus-5kmqc\" (UID: \"1a0a7a68-52cb-4beb-bb3e-1f5c00e4aa07\") " pod="openshift-multus/multus-5kmqc" Nov 04 03:50:03.279291 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278725 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-sys-fs\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:50:03.279291 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278747 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48e3e3bd-2ec1-4587-97b3-7b37fc5130fb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lrdgr\" (UID: \"48e3e3bd-2ec1-4587-97b3-7b37fc5130fb\") " pod="openshift-multus/multus-additional-cni-plugins-lrdgr" Nov 04 03:50:03.279291 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278935 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-systemd-units\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.279291 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.278963 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eaec1015-2b2b-4f06-b41d-028671919cf0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-htz59\" (UID: \"eaec1015-2b2b-4f06-b41d-028671919cf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htz59" Nov 04 03:50:03.279291 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.279007 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/42510c3d-0283-4901-9d23-a84793598ac9-log-socket\") pod \"ovnkube-node-zfgzh\" (UID: \"42510c3d-0283-4901-9d23-a84793598ac9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:03.279291 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:03.279048 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b1091f60-668b-4884-8548-fe4f4c438367-sys\") pod \"node-exporter-r7srf\" (UID: \"b1091f60-668b-4884-8548-fe4f4c438367\") " pod="openshift-monitoring/node-exporter-r7srf" Nov 04 03:50:04.214850 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:04.214798 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:50:04.306213 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:04.306182 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:04.553063 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:04.553028 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:04.569612 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:04.569586 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:04.973463 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:04.973387 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7976867cc6-jgp9n" Nov 04 03:50:06.258765 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:06.258715 6000 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-895vv_dde9ca65-5089-450e-97cd-89bf66adba30/konnectivity-agent/0.log" limit=1024 Nov 04 03:50:06.307194 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:06.307165 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-f92gv" Nov 04 03:50:06.356398 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:06.356367 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vcxpf" Nov 04 03:50:06.502525 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:06.502494 6000 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-895vv_dde9ca65-5089-450e-97cd-89bf66adba30/konnectivity-agent/0.log" limit=1024 Nov 04 03:50:06.712603 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:06.712572 6000 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-895vv_dde9ca65-5089-450e-97cd-89bf66adba30/konnectivity-agent/0.log" limit=1024 Nov 04 03:50:10.319335 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:10.319264 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:10.319713 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:10.319535 6000 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 03:50:12.886640 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:12.886603 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zfgzh" Nov 04 03:50:20.455158 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:20.455128 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-895vv" Nov 04 03:50:20.746359 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:50:20.746277 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 03:50:28.156892 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:28.156835 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:34.217599 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:34.217566 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-967tl" Nov 04 03:50:35.396848 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.396717 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 03:50:35.397362 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.397242 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="kube-rbac-proxy" containerID="cri-o://c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619" gracePeriod=600 Nov 04 03:50:35.397362 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.397292 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="thanos-sidecar" containerID="cri-o://0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907" gracePeriod=600 Nov 04 03:50:35.397488 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.397336 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="kube-rbac-proxy-web" containerID="cri-o://a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae" gracePeriod=600 Nov 04 03:50:35.397488 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.397238 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="prometheus" containerID="cri-o://ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3" gracePeriod=600 Nov 04 03:50:35.397488 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.397361 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="kube-rbac-proxy-thanos" containerID="cri-o://1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978" gracePeriod=600 Nov 04 03:50:35.397488 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.397346 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="config-reloader" containerID="cri-o://77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d" gracePeriod=600 Nov 04 03:50:35.686482 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.686448 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:35.822574 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.822531 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-metrics-client-certs\") pod \"e191ac4e-0058-44af-b530-5e68a054d946\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " Nov 04 03:50:35.822777 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.822583 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-kube-rbac-proxy\") pod \"e191ac4e-0058-44af-b530-5e68a054d946\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " Nov 04 03:50:35.822777 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.822617 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-db\") pod \"e191ac4e-0058-44af-b530-5e68a054d946\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " Nov 04 03:50:35.822777 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.822645 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-kubelet-serving-ca-bundle\") pod \"e191ac4e-0058-44af-b530-5e68a054d946\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " Nov 04 03:50:35.822777 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.822672 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-0\") pod \"e191ac4e-0058-44af-b530-5e68a054d946\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " Nov 04 03:50:35.822777 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.822707 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-config\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-config\") pod \"e191ac4e-0058-44af-b530-5e68a054d946\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " Nov 04 03:50:35.823054 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.822799 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-trusted-ca-bundle\") pod \"e191ac4e-0058-44af-b530-5e68a054d946\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " Nov 04 03:50:35.823054 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.822837 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-thanos-prometheus-http-client-file\") pod \"e191ac4e-0058-44af-b530-5e68a054d946\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " Nov 04 03:50:35.823054 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.822869 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-2\") pod \"e191ac4e-0058-44af-b530-5e68a054d946\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " Nov 04 03:50:35.823054 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.822900 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-1\") pod \"e191ac4e-0058-44af-b530-5e68a054d946\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " Nov 04 03:50:35.823054 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.822931 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-grpc-tls\") pod \"e191ac4e-0058-44af-b530-5e68a054d946\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " Nov 04 03:50:35.823054 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.822960 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-web-config\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-web-config\") pod \"e191ac4e-0058-44af-b530-5e68a054d946\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " Nov 04 03:50:35.823054 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.822992 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"e191ac4e-0058-44af-b530-5e68a054d946\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " Nov 04 03:50:35.823054 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.823017 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"e191ac4e-0058-44af-b530-5e68a054d946\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " Nov 04 03:50:35.823054 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.823049 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e191ac4e-0058-44af-b530-5e68a054d946-config-out\" (UniqueName: \"kubernetes.io/empty-dir/e191ac4e-0058-44af-b530-5e68a054d946-config-out\") pod \"e191ac4e-0058-44af-b530-5e68a054d946\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " Nov 04 03:50:35.823482 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.823077 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-metrics-client-ca\") pod \"e191ac4e-0058-44af-b530-5e68a054d946\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " Nov 04 03:50:35.823482 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.823112 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-serving-certs-ca-bundle\") pod \"e191ac4e-0058-44af-b530-5e68a054d946\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " Nov 04 03:50:35.823482 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.823165 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e191ac4e-0058-44af-b530-5e68a054d946-tls-assets\" (UniqueName: \"kubernetes.io/projected/e191ac4e-0058-44af-b530-5e68a054d946-tls-assets\") pod \"e191ac4e-0058-44af-b530-5e68a054d946\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " Nov 04 03:50:35.823482 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.823192 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e191ac4e-0058-44af-b530-5e68a054d946-kube-api-access-zlw9q\" (UniqueName: \"kubernetes.io/projected/e191ac4e-0058-44af-b530-5e68a054d946-kube-api-access-zlw9q\") pod \"e191ac4e-0058-44af-b530-5e68a054d946\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " Nov 04 03:50:35.823482 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.823223 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-tls\") pod \"e191ac4e-0058-44af-b530-5e68a054d946\" (UID: \"e191ac4e-0058-44af-b530-5e68a054d946\") " Nov 04 03:50:35.823482 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.823239 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-kubelet-serving-ca-bundle" pod "e191ac4e-0058-44af-b530-5e68a054d946" (UID: "e191ac4e-0058-44af-b530-5e68a054d946"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 03:50:35.823482 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.823319 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-2" pod "e191ac4e-0058-44af-b530-5e68a054d946" (UID: "e191ac4e-0058-44af-b530-5e68a054d946"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 03:50:35.823850 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.823516 6000 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:50:35.823850 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.823539 6000 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:50:35.823850 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.823590 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-trusted-ca-bundle" pod "e191ac4e-0058-44af-b530-5e68a054d946" (UID: "e191ac4e-0058-44af-b530-5e68a054d946"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 03:50:35.823850 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.823649 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-db" pod "e191ac4e-0058-44af-b530-5e68a054d946" (UID: "e191ac4e-0058-44af-b530-5e68a054d946"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 03:50:35.824583 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.824534 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-1" pod "e191ac4e-0058-44af-b530-5e68a054d946" (UID: "e191ac4e-0058-44af-b530-5e68a054d946"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 03:50:35.824726 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.824701 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-0" pod "e191ac4e-0058-44af-b530-5e68a054d946" (UID: "e191ac4e-0058-44af-b530-5e68a054d946"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 03:50:35.826212 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.826164 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-serving-certs-ca-bundle" pod "e191ac4e-0058-44af-b530-5e68a054d946" (UID: "e191ac4e-0058-44af-b530-5e68a054d946"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 03:50:35.826359 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.826315 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-config" pod "e191ac4e-0058-44af-b530-5e68a054d946" (UID: "e191ac4e-0058-44af-b530-5e68a054d946"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 03:50:35.827136 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.826873 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-metrics-client-ca" pod "e191ac4e-0058-44af-b530-5e68a054d946" (UID: "e191ac4e-0058-44af-b530-5e68a054d946"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 03:50:35.827968 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.827833 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-tls" pod "e191ac4e-0058-44af-b530-5e68a054d946" (UID: "e191ac4e-0058-44af-b530-5e68a054d946"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 03:50:35.827968 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.827904 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-thanos-sidecar-tls" pod "e191ac4e-0058-44af-b530-5e68a054d946" (UID: "e191ac4e-0058-44af-b530-5e68a054d946"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 03:50:35.828509 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.828474 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-kube-rbac-proxy-web" pod "e191ac4e-0058-44af-b530-5e68a054d946" (UID: "e191ac4e-0058-44af-b530-5e68a054d946"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 03:50:35.830793 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.830768 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-thanos-prometheus-http-client-file" pod "e191ac4e-0058-44af-b530-5e68a054d946" (UID: "e191ac4e-0058-44af-b530-5e68a054d946"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 03:50:35.831016 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.830767 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e191ac4e-0058-44af-b530-5e68a054d946-tls-assets" pod "e191ac4e-0058-44af-b530-5e68a054d946" (UID: "e191ac4e-0058-44af-b530-5e68a054d946"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 03:50:35.831131 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.830785 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e191ac4e-0058-44af-b530-5e68a054d946-config-out" pod "e191ac4e-0058-44af-b530-5e68a054d946" (UID: "e191ac4e-0058-44af-b530-5e68a054d946"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 03:50:35.831131 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.830975 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-metrics-client-certs" pod "e191ac4e-0058-44af-b530-5e68a054d946" (UID: "e191ac4e-0058-44af-b530-5e68a054d946"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 03:50:35.831131 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.831049 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-kube-rbac-proxy" pod "e191ac4e-0058-44af-b530-5e68a054d946" (UID: "e191ac4e-0058-44af-b530-5e68a054d946"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 03:50:35.831410 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.831389 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-grpc-tls" pod "e191ac4e-0058-44af-b530-5e68a054d946" (UID: "e191ac4e-0058-44af-b530-5e68a054d946"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 03:50:35.832130 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.832101 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e191ac4e-0058-44af-b530-5e68a054d946-kube-api-access-zlw9q" pod "e191ac4e-0058-44af-b530-5e68a054d946" (UID: "e191ac4e-0058-44af-b530-5e68a054d946"). InnerVolumeSpecName "kube-api-access-zlw9q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 03:50:35.840645 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.840609 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-web-config" pod "e191ac4e-0058-44af-b530-5e68a054d946" (UID: "e191ac4e-0058-44af-b530-5e68a054d946"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 03:50:35.924078 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.923984 6000 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-config\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:50:35.924078 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.924021 6000 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:50:35.924078 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.924037 6000 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:50:35.924078 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.924051 6000 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:50:35.924353 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.924093 6000 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-grpc-tls\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:50:35.924353 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.924105 6000 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-web-config\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:50:35.924353 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.924119 6000 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:50:35.924353 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.924131 6000 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:50:35.924353 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.924144 6000 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e191ac4e-0058-44af-b530-5e68a054d946-config-out\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:50:35.924353 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.924156 6000 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-metrics-client-ca\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:50:35.924353 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.924168 6000 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:50:35.924353 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.924182 6000 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e191ac4e-0058-44af-b530-5e68a054d946-tls-assets\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:50:35.924353 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.924197 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlw9q\" (UniqueName: \"kubernetes.io/projected/e191ac4e-0058-44af-b530-5e68a054d946-kube-api-access-zlw9q\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:50:35.924353 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.924209 6000 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:50:35.924353 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.924221 6000 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-metrics-client-certs\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:50:35.924353 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.924234 6000 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e191ac4e-0058-44af-b530-5e68a054d946-secret-kube-rbac-proxy\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:50:35.924353 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.924246 6000 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-db\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:50:35.924353 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:35.924259 6000 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e191ac4e-0058-44af-b530-5e68a054d946-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:50:36.257819 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.257714 6000 generic.go:356] "Generic (PLEG): container finished" podID="e191ac4e-0058-44af-b530-5e68a054d946" containerID="1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978" exitCode=0 Nov 04 03:50:36.257819 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.257743 6000 generic.go:356] "Generic (PLEG): container finished" podID="e191ac4e-0058-44af-b530-5e68a054d946" containerID="c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619" exitCode=0 Nov 04 03:50:36.257819 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.257750 6000 generic.go:356] "Generic (PLEG): container finished" podID="e191ac4e-0058-44af-b530-5e68a054d946" containerID="a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae" exitCode=0 Nov 04 03:50:36.257819 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.257778 6000 generic.go:356] "Generic (PLEG): container finished" podID="e191ac4e-0058-44af-b530-5e68a054d946" containerID="0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907" exitCode=0 Nov 04 03:50:36.257819 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.257784 6000 generic.go:356] "Generic (PLEG): container finished" podID="e191ac4e-0058-44af-b530-5e68a054d946" containerID="77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d" exitCode=0 Nov 04 03:50:36.257819 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.257789 6000 generic.go:356] "Generic (PLEG): container finished" podID="e191ac4e-0058-44af-b530-5e68a054d946" containerID="ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3" exitCode=0 Nov 04 03:50:36.257819 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.257795 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e191ac4e-0058-44af-b530-5e68a054d946","Type":"ContainerDied","Data":"1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978"} Nov 04 03:50:36.258191 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.257829 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.258191 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.257866 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e191ac4e-0058-44af-b530-5e68a054d946","Type":"ContainerDied","Data":"c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619"} Nov 04 03:50:36.258191 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.257894 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e191ac4e-0058-44af-b530-5e68a054d946","Type":"ContainerDied","Data":"a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae"} Nov 04 03:50:36.258191 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.257919 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e191ac4e-0058-44af-b530-5e68a054d946","Type":"ContainerDied","Data":"0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907"} Nov 04 03:50:36.258191 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.257934 6000 scope.go:122] "RemoveContainer" containerID="1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978" Nov 04 03:50:36.258191 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.257944 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e191ac4e-0058-44af-b530-5e68a054d946","Type":"ContainerDied","Data":"77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d"} Nov 04 03:50:36.258191 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.257969 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e191ac4e-0058-44af-b530-5e68a054d946","Type":"ContainerDied","Data":"ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3"} Nov 04 03:50:36.258191 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.257991 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e191ac4e-0058-44af-b530-5e68a054d946","Type":"ContainerDied","Data":"1694654274d75c17fa3418ca8df7950e1bf8a3fd72c1096b885492cd16ddf7a7"} Nov 04 03:50:36.265972 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.265949 6000 scope.go:122] "RemoveContainer" containerID="c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619" Nov 04 03:50:36.273336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.273316 6000 scope.go:122] "RemoveContainer" containerID="a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae" Nov 04 03:50:36.280136 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.280114 6000 scope.go:122] "RemoveContainer" containerID="0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907" Nov 04 03:50:36.281011 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.280989 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 03:50:36.286104 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.286079 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 03:50:36.288311 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.288290 6000 scope.go:122] "RemoveContainer" containerID="77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d" Nov 04 03:50:36.295258 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.295236 6000 scope.go:122] "RemoveContainer" containerID="ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3" Nov 04 03:50:36.302469 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.302447 6000 scope.go:122] "RemoveContainer" containerID="30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad" Nov 04 03:50:36.311572 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.311551 6000 scope.go:122] "RemoveContainer" containerID="1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978" Nov 04 03:50:36.311955 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:50:36.311928 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978\": container with ID starting with 1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978 not found: ID does not exist" containerID="1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978" Nov 04 03:50:36.312066 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.311967 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978"} err="failed to get container status \"1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978\": rpc error: code = NotFound desc = could not find container \"1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978\": container with ID starting with 1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978 not found: ID does not exist" Nov 04 03:50:36.312066 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.312004 6000 scope.go:122] "RemoveContainer" containerID="c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619" Nov 04 03:50:36.312273 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:50:36.312258 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619\": container with ID starting with c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619 not found: ID does not exist" containerID="c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619" Nov 04 03:50:36.312317 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.312275 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619"} err="failed to get container status \"c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619\": rpc error: code = NotFound desc = could not find container \"c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619\": container with ID starting with c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619 not found: ID does not exist" Nov 04 03:50:36.312317 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.312294 6000 scope.go:122] "RemoveContainer" containerID="a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae" Nov 04 03:50:36.312539 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:50:36.312517 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae\": container with ID starting with a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae not found: ID does not exist" containerID="a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae" Nov 04 03:50:36.312605 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.312550 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae"} err="failed to get container status \"a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae\": rpc error: code = NotFound desc = could not find container \"a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae\": container with ID starting with a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae not found: ID does not exist" Nov 04 03:50:36.312605 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.312575 6000 scope.go:122] "RemoveContainer" containerID="0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907" Nov 04 03:50:36.312829 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:50:36.312811 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907\": container with ID starting with 0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907 not found: ID does not exist" containerID="0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907" Nov 04 03:50:36.312874 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.312840 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907"} err="failed to get container status \"0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907\": rpc error: code = NotFound desc = could not find container \"0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907\": container with ID starting with 0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907 not found: ID does not exist" Nov 04 03:50:36.312874 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.312859 6000 scope.go:122] "RemoveContainer" containerID="77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d" Nov 04 03:50:36.313100 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:50:36.313082 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d\": container with ID starting with 77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d not found: ID does not exist" containerID="77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d" Nov 04 03:50:36.313158 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.313107 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d"} err="failed to get container status \"77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d\": rpc error: code = NotFound desc = could not find container \"77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d\": container with ID starting with 77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d not found: ID does not exist" Nov 04 03:50:36.313158 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.313125 6000 scope.go:122] "RemoveContainer" containerID="ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3" Nov 04 03:50:36.313361 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:50:36.313342 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3\": container with ID starting with ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3 not found: ID does not exist" containerID="ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3" Nov 04 03:50:36.313406 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.313366 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3"} err="failed to get container status \"ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3\": rpc error: code = NotFound desc = could not find container \"ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3\": container with ID starting with ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3 not found: ID does not exist" Nov 04 03:50:36.313406 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.313384 6000 scope.go:122] "RemoveContainer" containerID="30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad" Nov 04 03:50:36.313588 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:50:36.313570 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad\": container with ID starting with 30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad not found: ID does not exist" containerID="30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad" Nov 04 03:50:36.313649 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.313595 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad"} err="failed to get container status \"30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad\": rpc error: code = NotFound desc = could not find container \"30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad\": container with ID starting with 30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad not found: ID does not exist" Nov 04 03:50:36.313649 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.313612 6000 scope.go:122] "RemoveContainer" containerID="1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978" Nov 04 03:50:36.313860 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.313844 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978"} err="failed to get container status \"1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978\": rpc error: code = NotFound desc = could not find container \"1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978\": container with ID starting with 1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978 not found: ID does not exist" Nov 04 03:50:36.313915 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.313859 6000 scope.go:122] "RemoveContainer" containerID="c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619" Nov 04 03:50:36.314087 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.314059 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619"} err="failed to get container status \"c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619\": rpc error: code = NotFound desc = could not find container \"c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619\": container with ID starting with c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619 not found: ID does not exist" Nov 04 03:50:36.314087 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.314078 6000 scope.go:122] "RemoveContainer" containerID="a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae" Nov 04 03:50:36.314285 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.314266 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae"} err="failed to get container status \"a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae\": rpc error: code = NotFound desc = could not find container \"a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae\": container with ID starting with a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae not found: ID does not exist" Nov 04 03:50:36.314334 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.314286 6000 scope.go:122] "RemoveContainer" containerID="0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907" Nov 04 03:50:36.314499 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.314482 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907"} err="failed to get container status \"0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907\": rpc error: code = NotFound desc = could not find container \"0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907\": container with ID starting with 0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907 not found: ID does not exist" Nov 04 03:50:36.314569 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.314500 6000 scope.go:122] "RemoveContainer" containerID="77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d" Nov 04 03:50:36.314695 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.314678 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d"} err="failed to get container status \"77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d\": rpc error: code = NotFound desc = could not find container \"77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d\": container with ID starting with 77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d not found: ID does not exist" Nov 04 03:50:36.314741 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.314696 6000 scope.go:122] "RemoveContainer" containerID="ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3" Nov 04 03:50:36.314909 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.314891 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3"} err="failed to get container status \"ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3\": rpc error: code = NotFound desc = could not find container \"ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3\": container with ID starting with ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3 not found: ID does not exist" Nov 04 03:50:36.314979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.314909 6000 scope.go:122] "RemoveContainer" containerID="30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad" Nov 04 03:50:36.315117 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.315095 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad"} err="failed to get container status \"30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad\": rpc error: code = NotFound desc = could not find container \"30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad\": container with ID starting with 30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad not found: ID does not exist" Nov 04 03:50:36.315158 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.315117 6000 scope.go:122] "RemoveContainer" containerID="1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978" Nov 04 03:50:36.315308 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.315291 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978"} err="failed to get container status \"1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978\": rpc error: code = NotFound desc = could not find container \"1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978\": container with ID starting with 1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978 not found: ID does not exist" Nov 04 03:50:36.315356 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.315307 6000 scope.go:122] "RemoveContainer" containerID="c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619" Nov 04 03:50:36.315504 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.315487 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619"} err="failed to get container status \"c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619\": rpc error: code = NotFound desc = could not find container \"c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619\": container with ID starting with c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619 not found: ID does not exist" Nov 04 03:50:36.315565 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.315504 6000 scope.go:122] "RemoveContainer" containerID="a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae" Nov 04 03:50:36.315700 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.315685 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae"} err="failed to get container status \"a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae\": rpc error: code = NotFound desc = could not find container \"a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae\": container with ID starting with a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae not found: ID does not exist" Nov 04 03:50:36.315743 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.315700 6000 scope.go:122] "RemoveContainer" containerID="0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907" Nov 04 03:50:36.315940 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.315917 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907"} err="failed to get container status \"0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907\": rpc error: code = NotFound desc = could not find container \"0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907\": container with ID starting with 0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907 not found: ID does not exist" Nov 04 03:50:36.316006 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.315942 6000 scope.go:122] "RemoveContainer" containerID="77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d" Nov 04 03:50:36.316164 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.316144 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d"} err="failed to get container status \"77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d\": rpc error: code = NotFound desc = could not find container \"77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d\": container with ID starting with 77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d not found: ID does not exist" Nov 04 03:50:36.316269 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.316164 6000 scope.go:122] "RemoveContainer" containerID="ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3" Nov 04 03:50:36.316411 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.316391 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3"} err="failed to get container status \"ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3\": rpc error: code = NotFound desc = could not find container \"ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3\": container with ID starting with ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3 not found: ID does not exist" Nov 04 03:50:36.316450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.316412 6000 scope.go:122] "RemoveContainer" containerID="30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad" Nov 04 03:50:36.316633 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.316614 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad"} err="failed to get container status \"30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad\": rpc error: code = NotFound desc = could not find container \"30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad\": container with ID starting with 30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad not found: ID does not exist" Nov 04 03:50:36.316679 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.316634 6000 scope.go:122] "RemoveContainer" containerID="1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978" Nov 04 03:50:36.316898 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.316881 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978"} err="failed to get container status \"1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978\": rpc error: code = NotFound desc = could not find container \"1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978\": container with ID starting with 1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978 not found: ID does not exist" Nov 04 03:50:36.316946 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.316898 6000 scope.go:122] "RemoveContainer" containerID="c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619" Nov 04 03:50:36.317097 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.317081 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619"} err="failed to get container status \"c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619\": rpc error: code = NotFound desc = could not find container \"c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619\": container with ID starting with c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619 not found: ID does not exist" Nov 04 03:50:36.317136 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.317097 6000 scope.go:122] "RemoveContainer" containerID="a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae" Nov 04 03:50:36.317278 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.317259 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae"} err="failed to get container status \"a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae\": rpc error: code = NotFound desc = could not find container \"a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae\": container with ID starting with a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae not found: ID does not exist" Nov 04 03:50:36.317346 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.317278 6000 scope.go:122] "RemoveContainer" containerID="0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907" Nov 04 03:50:36.317512 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.317495 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907"} err="failed to get container status \"0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907\": rpc error: code = NotFound desc = could not find container \"0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907\": container with ID starting with 0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907 not found: ID does not exist" Nov 04 03:50:36.317561 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.317513 6000 scope.go:122] "RemoveContainer" containerID="77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d" Nov 04 03:50:36.317702 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.317682 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d"} err="failed to get container status \"77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d\": rpc error: code = NotFound desc = could not find container \"77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d\": container with ID starting with 77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d not found: ID does not exist" Nov 04 03:50:36.317766 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.317702 6000 scope.go:122] "RemoveContainer" containerID="ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3" Nov 04 03:50:36.317906 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.317888 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3"} err="failed to get container status \"ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3\": rpc error: code = NotFound desc = could not find container \"ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3\": container with ID starting with ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3 not found: ID does not exist" Nov 04 03:50:36.317953 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.317907 6000 scope.go:122] "RemoveContainer" containerID="30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad" Nov 04 03:50:36.318130 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.318112 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad"} err="failed to get container status \"30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad\": rpc error: code = NotFound desc = could not find container \"30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad\": container with ID starting with 30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad not found: ID does not exist" Nov 04 03:50:36.318177 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.318130 6000 scope.go:122] "RemoveContainer" containerID="1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978" Nov 04 03:50:36.318335 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.318308 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978"} err="failed to get container status \"1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978\": rpc error: code = NotFound desc = could not find container \"1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978\": container with ID starting with 1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978 not found: ID does not exist" Nov 04 03:50:36.318405 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.318333 6000 scope.go:122] "RemoveContainer" containerID="c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619" Nov 04 03:50:36.318571 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.318554 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619"} err="failed to get container status \"c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619\": rpc error: code = NotFound desc = could not find container \"c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619\": container with ID starting with c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619 not found: ID does not exist" Nov 04 03:50:36.318619 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.318571 6000 scope.go:122] "RemoveContainer" containerID="a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae" Nov 04 03:50:36.318782 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.318747 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae"} err="failed to get container status \"a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae\": rpc error: code = NotFound desc = could not find container \"a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae\": container with ID starting with a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae not found: ID does not exist" Nov 04 03:50:36.318829 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.318782 6000 scope.go:122] "RemoveContainer" containerID="0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907" Nov 04 03:50:36.318969 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.318948 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907"} err="failed to get container status \"0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907\": rpc error: code = NotFound desc = could not find container \"0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907\": container with ID starting with 0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907 not found: ID does not exist" Nov 04 03:50:36.319010 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.318969 6000 scope.go:122] "RemoveContainer" containerID="77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d" Nov 04 03:50:36.319146 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.319130 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d"} err="failed to get container status \"77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d\": rpc error: code = NotFound desc = could not find container \"77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d\": container with ID starting with 77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d not found: ID does not exist" Nov 04 03:50:36.319187 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.319147 6000 scope.go:122] "RemoveContainer" containerID="ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3" Nov 04 03:50:36.319341 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.319320 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3"} err="failed to get container status \"ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3\": rpc error: code = NotFound desc = could not find container \"ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3\": container with ID starting with ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3 not found: ID does not exist" Nov 04 03:50:36.319378 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.319342 6000 scope.go:122] "RemoveContainer" containerID="30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad" Nov 04 03:50:36.319520 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.319506 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad"} err="failed to get container status \"30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad\": rpc error: code = NotFound desc = could not find container \"30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad\": container with ID starting with 30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad not found: ID does not exist" Nov 04 03:50:36.319561 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.319520 6000 scope.go:122] "RemoveContainer" containerID="1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978" Nov 04 03:50:36.319710 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.319692 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978"} err="failed to get container status \"1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978\": rpc error: code = NotFound desc = could not find container \"1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978\": container with ID starting with 1cb8bec01631469b7e9d7c69f4c2a9fabb7ac2fc914fa13ea8532e49954fc978 not found: ID does not exist" Nov 04 03:50:36.319794 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.319711 6000 scope.go:122] "RemoveContainer" containerID="c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619" Nov 04 03:50:36.319928 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.319912 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619"} err="failed to get container status \"c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619\": rpc error: code = NotFound desc = could not find container \"c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619\": container with ID starting with c74836533da515b19484e2a275ae4718bd0156eac7cd50602280cf81a3e04619 not found: ID does not exist" Nov 04 03:50:36.319994 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.319929 6000 scope.go:122] "RemoveContainer" containerID="a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae" Nov 04 03:50:36.320152 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.320134 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae"} err="failed to get container status \"a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae\": rpc error: code = NotFound desc = could not find container \"a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae\": container with ID starting with a630e7e1b57a0c6c5a1eed74f4f2f70d7f03d810871337b6acc114f4eaffd8ae not found: ID does not exist" Nov 04 03:50:36.320216 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.320153 6000 scope.go:122] "RemoveContainer" containerID="0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907" Nov 04 03:50:36.320409 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.320390 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907"} err="failed to get container status \"0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907\": rpc error: code = NotFound desc = could not find container \"0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907\": container with ID starting with 0058459c74b27af0c2908fc4922313d4233ed3afc5f2390cb63b7f1d30767907 not found: ID does not exist" Nov 04 03:50:36.320471 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.320409 6000 scope.go:122] "RemoveContainer" containerID="77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d" Nov 04 03:50:36.320621 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.320601 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d"} err="failed to get container status \"77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d\": rpc error: code = NotFound desc = could not find container \"77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d\": container with ID starting with 77506e9b0afd427e56706facd7b6dd5cd1857e2d5b129a9849ea59d8a76acf3d not found: ID does not exist" Nov 04 03:50:36.320671 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.320622 6000 scope.go:122] "RemoveContainer" containerID="ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3" Nov 04 03:50:36.320855 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.320836 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3"} err="failed to get container status \"ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3\": rpc error: code = NotFound desc = could not find container \"ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3\": container with ID starting with ef05f213c32767455e4b1a4b8d9ff4e660e933545d6a20e9fd4800512c8c6bb3 not found: ID does not exist" Nov 04 03:50:36.320929 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.320856 6000 scope.go:122] "RemoveContainer" containerID="30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad" Nov 04 03:50:36.321091 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.321073 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad"} err="failed to get container status \"30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad\": rpc error: code = NotFound desc = could not find container \"30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad\": container with ID starting with 30e583f585db016d8b88468c5f90280b7c7ef234c6f97314ae5c2d73db1878ad not found: ID does not exist" Nov 04 03:50:36.322235 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.322113 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 03:50:36.322425 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.322410 6000 cpu_manager.go:414] "containerMap: removing container" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="prometheus" Nov 04 03:50:36.322469 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.322427 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="prometheus" Nov 04 03:50:36.322469 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.322439 6000 cpu_manager.go:414] "containerMap: removing container" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="kube-rbac-proxy-thanos" Nov 04 03:50:36.322469 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.322444 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="kube-rbac-proxy-thanos" Nov 04 03:50:36.322469 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.322460 6000 cpu_manager.go:414] "containerMap: removing container" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="kube-rbac-proxy-web" Nov 04 03:50:36.322469 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.322467 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="kube-rbac-proxy-web" Nov 04 03:50:36.322632 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.322477 6000 cpu_manager.go:414] "containerMap: removing container" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="thanos-sidecar" Nov 04 03:50:36.322632 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.322482 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="thanos-sidecar" Nov 04 03:50:36.322632 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.322489 6000 cpu_manager.go:414] "containerMap: removing container" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="kube-rbac-proxy" Nov 04 03:50:36.322632 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.322493 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="kube-rbac-proxy" Nov 04 03:50:36.322632 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.322506 6000 cpu_manager.go:414] "containerMap: removing container" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="init-config-reloader" Nov 04 03:50:36.322632 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.322511 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="init-config-reloader" Nov 04 03:50:36.322632 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.322520 6000 cpu_manager.go:414] "containerMap: removing container" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="config-reloader" Nov 04 03:50:36.322632 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.322525 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="config-reloader" Nov 04 03:50:36.322632 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.322574 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="init-config-reloader" Nov 04 03:50:36.322632 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.322583 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="config-reloader" Nov 04 03:50:36.322632 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.322589 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="prometheus" Nov 04 03:50:36.322632 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.322594 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="kube-rbac-proxy-thanos" Nov 04 03:50:36.322632 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.322603 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="kube-rbac-proxy-web" Nov 04 03:50:36.322632 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.322611 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="thanos-sidecar" Nov 04 03:50:36.322632 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.322616 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="e191ac4e-0058-44af-b530-5e68a054d946" containerName="kube-rbac-proxy" Nov 04 03:50:36.358219 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.358189 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.361341 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.361313 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 03:50:36.367384 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.367356 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Nov 04 03:50:36.367511 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.367428 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Nov 04 03:50:36.368113 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.368089 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-bjr97m2gdpk1p\"" Nov 04 03:50:36.368113 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.368091 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Nov 04 03:50:36.368313 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.368137 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Nov 04 03:50:36.368313 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.368150 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Nov 04 03:50:36.368313 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.368228 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Nov 04 03:50:36.368465 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.368370 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bsczw\"" Nov 04 03:50:36.368465 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.368411 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Nov 04 03:50:36.368746 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.368704 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Nov 04 03:50:36.368746 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.368718 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Nov 04 03:50:36.368746 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.368720 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Nov 04 03:50:36.368746 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.368743 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Nov 04 03:50:36.368907 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.368710 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Nov 04 03:50:36.377836 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.377812 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Nov 04 03:50:36.377836 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.377817 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Nov 04 03:50:36.378013 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.377953 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Nov 04 03:50:36.427839 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.427796 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.428219 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.427908 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.428219 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.427938 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.428219 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.427959 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/38fe00d5-85df-48ec-8611-1a7eb735a845-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.428219 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.427980 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-config\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.428219 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.428038 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.428219 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.428119 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38fe00d5-85df-48ec-8611-1a7eb735a845-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.428219 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.428145 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.428219 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.428162 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/38fe00d5-85df-48ec-8611-1a7eb735a845-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.428219 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.428180 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/38fe00d5-85df-48ec-8611-1a7eb735a845-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.428487 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.428223 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38fe00d5-85df-48ec-8611-1a7eb735a845-config-out\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.428487 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.428267 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38fe00d5-85df-48ec-8611-1a7eb735a845-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.428487 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.428309 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38fe00d5-85df-48ec-8611-1a7eb735a845-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.428487 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.428375 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38fe00d5-85df-48ec-8611-1a7eb735a845-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.428487 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.428425 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38fe00d5-85df-48ec-8611-1a7eb735a845-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.428487 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.428440 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.428487 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.428454 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.428487 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.428474 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-web-config\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.428702 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.428550 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb57g\" (UniqueName: \"kubernetes.io/projected/38fe00d5-85df-48ec-8611-1a7eb735a845-kube-api-access-wb57g\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.428702 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.428576 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/38fe00d5-85df-48ec-8611-1a7eb735a845-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.529602 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.529508 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/38fe00d5-85df-48ec-8611-1a7eb735a845-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.529602 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.529549 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.529602 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.529565 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.529602 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.529583 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.529602 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.529603 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/38fe00d5-85df-48ec-8611-1a7eb735a845-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.530010 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.529622 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-config\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.530010 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.529652 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.530010 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.529891 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38fe00d5-85df-48ec-8611-1a7eb735a845-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.530010 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.529923 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.530010 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.529955 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/38fe00d5-85df-48ec-8611-1a7eb735a845-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.530010 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.529977 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/38fe00d5-85df-48ec-8611-1a7eb735a845-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.530010 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.529994 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38fe00d5-85df-48ec-8611-1a7eb735a845-config-out\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.530412 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.530022 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38fe00d5-85df-48ec-8611-1a7eb735a845-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.530412 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.530059 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38fe00d5-85df-48ec-8611-1a7eb735a845-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.530412 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.530064 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/38fe00d5-85df-48ec-8611-1a7eb735a845-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.530412 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.530090 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38fe00d5-85df-48ec-8611-1a7eb735a845-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.530412 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.530063 6000 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Nov 04 03:50:36.530412 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.530126 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38fe00d5-85df-48ec-8611-1a7eb735a845-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.530412 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.530153 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.530412 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.530177 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.530412 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.530200 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-web-config\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.530412 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.530253 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb57g\" (UniqueName: \"kubernetes.io/projected/38fe00d5-85df-48ec-8611-1a7eb735a845-kube-api-access-wb57g\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.530939 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.530477 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/38fe00d5-85df-48ec-8611-1a7eb735a845-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.530939 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.530580 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38fe00d5-85df-48ec-8611-1a7eb735a845-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.531055 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.531004 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/38fe00d5-85df-48ec-8611-1a7eb735a845-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.531329 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.531302 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38fe00d5-85df-48ec-8611-1a7eb735a845-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.532841 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.531824 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38fe00d5-85df-48ec-8611-1a7eb735a845-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.532841 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.532511 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38fe00d5-85df-48ec-8611-1a7eb735a845-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.533404 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.533378 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.534101 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.534076 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38fe00d5-85df-48ec-8611-1a7eb735a845-config-out\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.534185 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.534103 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/38fe00d5-85df-48ec-8611-1a7eb735a845-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.534517 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.534492 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.534917 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.534892 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.535878 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.535854 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.535956 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.535883 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38fe00d5-85df-48ec-8611-1a7eb735a845-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.536097 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.536070 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-web-config\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.536401 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.536379 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.536556 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.536529 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.536602 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.536569 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-config\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.536658 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.536643 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/38fe00d5-85df-48ec-8611-1a7eb735a845-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.543233 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.543207 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb57g\" (UniqueName: \"kubernetes.io/projected/38fe00d5-85df-48ec-8611-1a7eb735a845-kube-api-access-wb57g\") pod \"prometheus-k8s-0\" (UID: \"38fe00d5-85df-48ec-8611-1a7eb735a845\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:36.770171 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:36.770136 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:37.086934 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:37.086911 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 03:50:37.088937 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:50:37.088906 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38fe00d5_85df_48ec_8611_1a7eb735a845.slice/crio-8be4c91fd1f1565105f0564a28b60ba372dddf81d8347f16e8099437bc04dac1 WatchSource:0}: Error finding container 8be4c91fd1f1565105f0564a28b60ba372dddf81d8347f16e8099437bc04dac1: Status 404 returned error can't find the container with id 8be4c91fd1f1565105f0564a28b60ba372dddf81d8347f16e8099437bc04dac1 Nov 04 03:50:37.264341 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:37.264301 6000 generic.go:356] "Generic (PLEG): container finished" podID="38fe00d5-85df-48ec-8611-1a7eb735a845" containerID="c512831a8fbc7743c15db6bf06a0c04a5149e347e4accba9a3ecdcfb3a73cc2a" exitCode=0 Nov 04 03:50:37.264519 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:37.264358 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38fe00d5-85df-48ec-8611-1a7eb735a845","Type":"ContainerDied","Data":"c512831a8fbc7743c15db6bf06a0c04a5149e347e4accba9a3ecdcfb3a73cc2a"} Nov 04 03:50:37.264519 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:37.264434 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38fe00d5-85df-48ec-8611-1a7eb735a845","Type":"ContainerStarted","Data":"8be4c91fd1f1565105f0564a28b60ba372dddf81d8347f16e8099437bc04dac1"} Nov 04 03:50:38.079918 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:38.079885 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e191ac4e-0058-44af-b530-5e68a054d946" path="/var/lib/kubelet/pods/e191ac4e-0058-44af-b530-5e68a054d946/volumes" Nov 04 03:50:38.275718 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:38.275683 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38fe00d5-85df-48ec-8611-1a7eb735a845","Type":"ContainerStarted","Data":"551164361de5fd04d1da9a717073d050cc5750e2dc6228e8821a7412976491d9"} Nov 04 03:50:38.275889 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:38.275788 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38fe00d5-85df-48ec-8611-1a7eb735a845","Type":"ContainerStarted","Data":"3e84b719fac2b2a1dce26c159f6c9da3ad3cc94f4118e4fad15a0b01dee485ca"} Nov 04 03:50:38.275889 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:38.275816 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38fe00d5-85df-48ec-8611-1a7eb735a845","Type":"ContainerStarted","Data":"76ab45d23512ba78a6cc18c0349ac41fb83b88578ef0034198127af8d2f7c5c7"} Nov 04 03:50:38.275889 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:38.275841 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38fe00d5-85df-48ec-8611-1a7eb735a845","Type":"ContainerStarted","Data":"35e4e8b763011992528d81f5d50f00e28e2048e546b2d4c43128639cab7c3aac"} Nov 04 03:50:38.275889 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:38.275865 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38fe00d5-85df-48ec-8611-1a7eb735a845","Type":"ContainerStarted","Data":"487fe70b91b26939e628d5aa4028b95ae9caa48e8a57e1aa78ace417fbfab845"} Nov 04 03:50:38.275889 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:38.275888 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38fe00d5-85df-48ec-8611-1a7eb735a845","Type":"ContainerStarted","Data":"3fc6e2237f343d3731fd05948f1424d226f2876f978b3d6ac8d4d76d046db7fa"} Nov 04 03:50:38.317542 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:38.317455 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.30132792 podStartE2EDuration="2.317437432s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.109512ms" isStatefulPod=false podCreationTimestamp="2026-11-04 03:50:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:50:38.315674177 +0000 UTC m=+36.354114140" watchObservedRunningTime="2026-11-04 03:50:38.317437432 +0000 UTC m=+36.355877394" Nov 04 03:50:41.770403 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:50:41.770358 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:50:50.747633 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:50:50.747591 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 03:51:20.747169 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:51:20.747129 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 03:51:26.126573 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:51:26.126526 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:51:26.142276 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:51:26.142243 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:51:26.452488 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:51:26.452420 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 03:51:50.747018 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:51:50.746920 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 03:52:20.746935 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:52:20.746895 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 03:52:50.746571 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:52:50.746532 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 03:52:52.016336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.016299 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5"] Nov 04 03:52:52.019541 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.019522 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5" Nov 04 03:52:52.023436 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.023408 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Nov 04 03:52:52.024786 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.024747 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Nov 04 03:52:52.025000 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.024983 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-hspm7\"" Nov 04 03:52:52.029286 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.029261 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5"] Nov 04 03:52:52.111934 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.111894 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq"] Nov 04 03:52:52.115328 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.115304 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq" Nov 04 03:52:52.118199 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.118171 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnxh9\" (UniqueName: \"kubernetes.io/projected/b27681f5-ec3b-4fcf-bcaa-4782fb333b88-kube-api-access-hnxh9\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5\" (UID: \"b27681f5-ec3b-4fcf-bcaa-4782fb333b88\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5" Nov 04 03:52:52.118340 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.118218 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b27681f5-ec3b-4fcf-bcaa-4782fb333b88-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5\" (UID: \"b27681f5-ec3b-4fcf-bcaa-4782fb333b88\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5" Nov 04 03:52:52.118340 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.118237 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/639c38c2-be7f-4e49-bc66-8ff5a640b6c5-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq\" (UID: \"639c38c2-be7f-4e49-bc66-8ff5a640b6c5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq" Nov 04 03:52:52.118455 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.118355 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b27681f5-ec3b-4fcf-bcaa-4782fb333b88-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5\" (UID: \"b27681f5-ec3b-4fcf-bcaa-4782fb333b88\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5" Nov 04 03:52:52.118455 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.118408 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ns56\" (UniqueName: \"kubernetes.io/projected/639c38c2-be7f-4e49-bc66-8ff5a640b6c5-kube-api-access-7ns56\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq\" (UID: \"639c38c2-be7f-4e49-bc66-8ff5a640b6c5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq" Nov 04 03:52:52.118562 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.118515 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/639c38c2-be7f-4e49-bc66-8ff5a640b6c5-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq\" (UID: \"639c38c2-be7f-4e49-bc66-8ff5a640b6c5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq" Nov 04 03:52:52.119198 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.119183 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-h22ls\"" Nov 04 03:52:52.124404 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.124381 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq"] Nov 04 03:52:52.216322 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.216289 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj"] Nov 04 03:52:52.218980 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.218942 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnxh9\" (UniqueName: \"kubernetes.io/projected/b27681f5-ec3b-4fcf-bcaa-4782fb333b88-kube-api-access-hnxh9\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5\" (UID: \"b27681f5-ec3b-4fcf-bcaa-4782fb333b88\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5" Nov 04 03:52:52.219134 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.218993 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b27681f5-ec3b-4fcf-bcaa-4782fb333b88-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5\" (UID: \"b27681f5-ec3b-4fcf-bcaa-4782fb333b88\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5" Nov 04 03:52:52.219204 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.219149 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/639c38c2-be7f-4e49-bc66-8ff5a640b6c5-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq\" (UID: \"639c38c2-be7f-4e49-bc66-8ff5a640b6c5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq" Nov 04 03:52:52.219262 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.219214 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b27681f5-ec3b-4fcf-bcaa-4782fb333b88-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5\" (UID: \"b27681f5-ec3b-4fcf-bcaa-4782fb333b88\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5" Nov 04 03:52:52.219323 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.219259 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ns56\" (UniqueName: \"kubernetes.io/projected/639c38c2-be7f-4e49-bc66-8ff5a640b6c5-kube-api-access-7ns56\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq\" (UID: \"639c38c2-be7f-4e49-bc66-8ff5a640b6c5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq" Nov 04 03:52:52.219323 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.219302 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/639c38c2-be7f-4e49-bc66-8ff5a640b6c5-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq\" (UID: \"639c38c2-be7f-4e49-bc66-8ff5a640b6c5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq" Nov 04 03:52:52.219434 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.219329 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b27681f5-ec3b-4fcf-bcaa-4782fb333b88-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5\" (UID: \"b27681f5-ec3b-4fcf-bcaa-4782fb333b88\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5" Nov 04 03:52:52.219540 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.219517 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b27681f5-ec3b-4fcf-bcaa-4782fb333b88-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5\" (UID: \"b27681f5-ec3b-4fcf-bcaa-4782fb333b88\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5" Nov 04 03:52:52.219593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.219547 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/639c38c2-be7f-4e49-bc66-8ff5a640b6c5-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq\" (UID: \"639c38c2-be7f-4e49-bc66-8ff5a640b6c5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq" Nov 04 03:52:52.219593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.219578 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj" Nov 04 03:52:52.219593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.219586 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/639c38c2-be7f-4e49-bc66-8ff5a640b6c5-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq\" (UID: \"639c38c2-be7f-4e49-bc66-8ff5a640b6c5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq" Nov 04 03:52:52.223584 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.223561 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-2blc5\"" Nov 04 03:52:52.229420 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.229391 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj"] Nov 04 03:52:52.232584 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.232556 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnxh9\" (UniqueName: \"kubernetes.io/projected/b27681f5-ec3b-4fcf-bcaa-4782fb333b88-kube-api-access-hnxh9\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5\" (UID: \"b27681f5-ec3b-4fcf-bcaa-4782fb333b88\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5" Nov 04 03:52:52.232718 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.232645 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ns56\" (UniqueName: \"kubernetes.io/projected/639c38c2-be7f-4e49-bc66-8ff5a640b6c5-kube-api-access-7ns56\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq\" (UID: \"639c38c2-be7f-4e49-bc66-8ff5a640b6c5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq" Nov 04 03:52:52.320275 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.320238 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41e4fa82-143e-415e-9549-b859194077ec-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj\" (UID: \"41e4fa82-143e-415e-9549-b859194077ec\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj" Nov 04 03:52:52.320459 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.320309 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41e4fa82-143e-415e-9549-b859194077ec-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj\" (UID: \"41e4fa82-143e-415e-9549-b859194077ec\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj" Nov 04 03:52:52.320459 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.320380 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lztkk\" (UniqueName: \"kubernetes.io/projected/41e4fa82-143e-415e-9549-b859194077ec-kube-api-access-lztkk\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj\" (UID: \"41e4fa82-143e-415e-9549-b859194077ec\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj" Nov 04 03:52:52.326481 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.326448 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc"] Nov 04 03:52:52.329869 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.329844 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc" Nov 04 03:52:52.335435 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.335402 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5" Nov 04 03:52:52.354853 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.354820 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-gsbt4\"" Nov 04 03:52:52.358573 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.358545 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc"] Nov 04 03:52:52.418501 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.418469 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x"] Nov 04 03:52:52.421277 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.421246 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7155f85-0a15-4f1e-aa99-8c51b53f6b61-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc\" (UID: \"c7155f85-0a15-4f1e-aa99-8c51b53f6b61\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc" Nov 04 03:52:52.421427 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.421288 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41e4fa82-143e-415e-9549-b859194077ec-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj\" (UID: \"41e4fa82-143e-415e-9549-b859194077ec\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj" Nov 04 03:52:52.421427 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.421311 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcvs9\" (UniqueName: \"kubernetes.io/projected/c7155f85-0a15-4f1e-aa99-8c51b53f6b61-kube-api-access-dcvs9\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc\" (UID: \"c7155f85-0a15-4f1e-aa99-8c51b53f6b61\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc" Nov 04 03:52:52.421427 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.421404 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7155f85-0a15-4f1e-aa99-8c51b53f6b61-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc\" (UID: \"c7155f85-0a15-4f1e-aa99-8c51b53f6b61\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc" Nov 04 03:52:52.421596 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.421443 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lztkk\" (UniqueName: \"kubernetes.io/projected/41e4fa82-143e-415e-9549-b859194077ec-kube-api-access-lztkk\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj\" (UID: \"41e4fa82-143e-415e-9549-b859194077ec\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj" Nov 04 03:52:52.421596 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.421484 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41e4fa82-143e-415e-9549-b859194077ec-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj\" (UID: \"41e4fa82-143e-415e-9549-b859194077ec\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj" Nov 04 03:52:52.421796 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.421774 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41e4fa82-143e-415e-9549-b859194077ec-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj\" (UID: \"41e4fa82-143e-415e-9549-b859194077ec\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj" Nov 04 03:52:52.421878 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.421800 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41e4fa82-143e-415e-9549-b859194077ec-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj\" (UID: \"41e4fa82-143e-415e-9549-b859194077ec\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj" Nov 04 03:52:52.421946 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.421933 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x" Nov 04 03:52:52.431171 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.431132 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq" Nov 04 03:52:52.431171 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.431154 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x"] Nov 04 03:52:52.434935 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.434909 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lztkk\" (UniqueName: \"kubernetes.io/projected/41e4fa82-143e-415e-9549-b859194077ec-kube-api-access-lztkk\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj\" (UID: \"41e4fa82-143e-415e-9549-b859194077ec\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj" Nov 04 03:52:52.439156 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.439130 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-2zt2c\"" Nov 04 03:52:52.522439 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.522402 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcvs9\" (UniqueName: \"kubernetes.io/projected/c7155f85-0a15-4f1e-aa99-8c51b53f6b61-kube-api-access-dcvs9\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc\" (UID: \"c7155f85-0a15-4f1e-aa99-8c51b53f6b61\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc" Nov 04 03:52:52.522439 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.522444 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7155f85-0a15-4f1e-aa99-8c51b53f6b61-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc\" (UID: \"c7155f85-0a15-4f1e-aa99-8c51b53f6b61\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc" Nov 04 03:52:52.522662 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.522465 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrktw\" (UniqueName: \"kubernetes.io/projected/eb4f9c7b-e5ae-4aed-af9d-d092cb13add9-kube-api-access-lrktw\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x\" (UID: \"eb4f9c7b-e5ae-4aed-af9d-d092cb13add9\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x" Nov 04 03:52:52.522662 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.522517 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9c7b-e5ae-4aed-af9d-d092cb13add9-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x\" (UID: \"eb4f9c7b-e5ae-4aed-af9d-d092cb13add9\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x" Nov 04 03:52:52.522662 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.522540 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7155f85-0a15-4f1e-aa99-8c51b53f6b61-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc\" (UID: \"c7155f85-0a15-4f1e-aa99-8c51b53f6b61\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc" Nov 04 03:52:52.522662 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.522560 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9c7b-e5ae-4aed-af9d-d092cb13add9-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x\" (UID: \"eb4f9c7b-e5ae-4aed-af9d-d092cb13add9\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x" Nov 04 03:52:52.522855 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.522830 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7155f85-0a15-4f1e-aa99-8c51b53f6b61-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc\" (UID: \"c7155f85-0a15-4f1e-aa99-8c51b53f6b61\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc" Nov 04 03:52:52.522900 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.522881 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7155f85-0a15-4f1e-aa99-8c51b53f6b61-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc\" (UID: \"c7155f85-0a15-4f1e-aa99-8c51b53f6b61\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc" Nov 04 03:52:52.535238 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.535206 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcvs9\" (UniqueName: \"kubernetes.io/projected/c7155f85-0a15-4f1e-aa99-8c51b53f6b61-kube-api-access-dcvs9\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc\" (UID: \"c7155f85-0a15-4f1e-aa99-8c51b53f6b61\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc" Nov 04 03:52:52.535351 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.535331 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj" Nov 04 03:52:52.614429 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.614396 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5"] Nov 04 03:52:52.617794 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:52:52.617764 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb27681f5_ec3b_4fcf_bcaa_4782fb333b88.slice/crio-e77a065fb20e9cb0ad443360f76e8577b1bfaa55224bf793050cf3eebd99352b WatchSource:0}: Error finding container e77a065fb20e9cb0ad443360f76e8577b1bfaa55224bf793050cf3eebd99352b: Status 404 returned error can't find the container with id e77a065fb20e9cb0ad443360f76e8577b1bfaa55224bf793050cf3eebd99352b Nov 04 03:52:52.619857 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.619838 6000 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 03:52:52.622985 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.622963 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrktw\" (UniqueName: \"kubernetes.io/projected/eb4f9c7b-e5ae-4aed-af9d-d092cb13add9-kube-api-access-lrktw\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x\" (UID: \"eb4f9c7b-e5ae-4aed-af9d-d092cb13add9\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x" Nov 04 03:52:52.623074 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.623058 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9c7b-e5ae-4aed-af9d-d092cb13add9-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x\" (UID: \"eb4f9c7b-e5ae-4aed-af9d-d092cb13add9\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x" Nov 04 03:52:52.623132 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.623106 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9c7b-e5ae-4aed-af9d-d092cb13add9-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x\" (UID: \"eb4f9c7b-e5ae-4aed-af9d-d092cb13add9\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x" Nov 04 03:52:52.623438 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.623418 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9c7b-e5ae-4aed-af9d-d092cb13add9-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x\" (UID: \"eb4f9c7b-e5ae-4aed-af9d-d092cb13add9\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x" Nov 04 03:52:52.623550 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.623439 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9c7b-e5ae-4aed-af9d-d092cb13add9-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x\" (UID: \"eb4f9c7b-e5ae-4aed-af9d-d092cb13add9\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x" Nov 04 03:52:52.635921 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.635884 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrktw\" (UniqueName: \"kubernetes.io/projected/eb4f9c7b-e5ae-4aed-af9d-d092cb13add9-kube-api-access-lrktw\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x\" (UID: \"eb4f9c7b-e5ae-4aed-af9d-d092cb13add9\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x" Nov 04 03:52:52.646678 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.646648 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc" Nov 04 03:52:52.709294 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.709214 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq"] Nov 04 03:52:52.712628 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:52:52.712599 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod639c38c2_be7f_4e49_bc66_8ff5a640b6c5.slice/crio-ce65697dcd258d22b2fc91267d5ef75caf17fc99d4abbed3a7881d3dcd5e6c89 WatchSource:0}: Error finding container ce65697dcd258d22b2fc91267d5ef75caf17fc99d4abbed3a7881d3dcd5e6c89: Status 404 returned error can't find the container with id ce65697dcd258d22b2fc91267d5ef75caf17fc99d4abbed3a7881d3dcd5e6c89 Nov 04 03:52:52.716976 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.716949 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5" event={"ID":"b27681f5-ec3b-4fcf-bcaa-4782fb333b88","Type":"ContainerStarted","Data":"e77a065fb20e9cb0ad443360f76e8577b1bfaa55224bf793050cf3eebd99352b"} Nov 04 03:52:52.742136 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.742098 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x" Nov 04 03:52:52.812235 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.812207 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj"] Nov 04 03:52:52.816428 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:52:52.816390 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41e4fa82_143e_415e_9549_b859194077ec.slice/crio-a8f3940070e4e385fd3f03612e1085e0ea44ce269206470f4ffd0e388ff04320 WatchSource:0}: Error finding container a8f3940070e4e385fd3f03612e1085e0ea44ce269206470f4ffd0e388ff04320: Status 404 returned error can't find the container with id a8f3940070e4e385fd3f03612e1085e0ea44ce269206470f4ffd0e388ff04320 Nov 04 03:52:52.920603 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:52.920467 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc"] Nov 04 03:52:52.923352 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:52:52.923324 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7155f85_0a15_4f1e_aa99_8c51b53f6b61.slice/crio-545c2758ff09e24cf1dc51a98f01f271586e91b147e9ef543ca58e5aaba99411 WatchSource:0}: Error finding container 545c2758ff09e24cf1dc51a98f01f271586e91b147e9ef543ca58e5aaba99411: Status 404 returned error can't find the container with id 545c2758ff09e24cf1dc51a98f01f271586e91b147e9ef543ca58e5aaba99411 Nov 04 03:52:53.015704 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:53.015672 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x"] Nov 04 03:52:53.018517 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:52:53.018490 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb4f9c7b_e5ae_4aed_af9d_d092cb13add9.slice/crio-a4187619610df9759e5cd6bd05cc252e901b79f751695a75d8d0484b35b9f1c6 WatchSource:0}: Error finding container a4187619610df9759e5cd6bd05cc252e901b79f751695a75d8d0484b35b9f1c6: Status 404 returned error can't find the container with id a4187619610df9759e5cd6bd05cc252e901b79f751695a75d8d0484b35b9f1c6 Nov 04 03:52:53.724141 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:53.724094 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x" event={"ID":"eb4f9c7b-e5ae-4aed-af9d-d092cb13add9","Type":"ContainerStarted","Data":"a4187619610df9759e5cd6bd05cc252e901b79f751695a75d8d0484b35b9f1c6"} Nov 04 03:52:53.728060 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:53.728027 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc" event={"ID":"c7155f85-0a15-4f1e-aa99-8c51b53f6b61","Type":"ContainerStarted","Data":"545c2758ff09e24cf1dc51a98f01f271586e91b147e9ef543ca58e5aaba99411"} Nov 04 03:52:53.729986 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:53.729958 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj" event={"ID":"41e4fa82-143e-415e-9549-b859194077ec","Type":"ContainerStarted","Data":"a8f3940070e4e385fd3f03612e1085e0ea44ce269206470f4ffd0e388ff04320"} Nov 04 03:52:53.734255 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:53.734222 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq" event={"ID":"639c38c2-be7f-4e49-bc66-8ff5a640b6c5","Type":"ContainerStarted","Data":"ce65697dcd258d22b2fc91267d5ef75caf17fc99d4abbed3a7881d3dcd5e6c89"} Nov 04 03:52:54.579836 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:54.578809 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7"] Nov 04 03:52:54.583181 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:54.583154 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7" Nov 04 03:52:54.588799 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:54.588715 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-bd9pl\"" Nov 04 03:52:54.591479 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:54.591326 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7"] Nov 04 03:52:54.642394 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:54.642123 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7h6t\" (UniqueName: \"kubernetes.io/projected/c070260d-4185-49d8-8467-af038460ca9f-kube-api-access-t7h6t\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7\" (UID: \"c070260d-4185-49d8-8467-af038460ca9f\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7" Nov 04 03:52:54.642394 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:54.642172 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c070260d-4185-49d8-8467-af038460ca9f-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7\" (UID: \"c070260d-4185-49d8-8467-af038460ca9f\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7" Nov 04 03:52:54.642394 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:54.642204 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c070260d-4185-49d8-8467-af038460ca9f-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7\" (UID: \"c070260d-4185-49d8-8467-af038460ca9f\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7" Nov 04 03:52:54.744749 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:54.743539 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7h6t\" (UniqueName: \"kubernetes.io/projected/c070260d-4185-49d8-8467-af038460ca9f-kube-api-access-t7h6t\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7\" (UID: \"c070260d-4185-49d8-8467-af038460ca9f\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7" Nov 04 03:52:54.744749 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:54.743601 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c070260d-4185-49d8-8467-af038460ca9f-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7\" (UID: \"c070260d-4185-49d8-8467-af038460ca9f\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7" Nov 04 03:52:54.744749 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:54.743625 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c070260d-4185-49d8-8467-af038460ca9f-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7\" (UID: \"c070260d-4185-49d8-8467-af038460ca9f\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7" Nov 04 03:52:54.744749 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:54.744121 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c070260d-4185-49d8-8467-af038460ca9f-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7\" (UID: \"c070260d-4185-49d8-8467-af038460ca9f\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7" Nov 04 03:52:54.744749 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:54.744704 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c070260d-4185-49d8-8467-af038460ca9f-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7\" (UID: \"c070260d-4185-49d8-8467-af038460ca9f\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7" Nov 04 03:52:54.764342 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:54.764285 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7h6t\" (UniqueName: \"kubernetes.io/projected/c070260d-4185-49d8-8467-af038460ca9f-kube-api-access-t7h6t\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7\" (UID: \"c070260d-4185-49d8-8467-af038460ca9f\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7" Nov 04 03:52:54.907051 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:54.906970 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7" Nov 04 03:52:55.215175 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:55.215011 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7"] Nov 04 03:52:55.778382 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:55.778334 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7" event={"ID":"c070260d-4185-49d8-8467-af038460ca9f","Type":"ContainerStarted","Data":"00d913251e6be6f81044985ef22be2894e48c060b27a411b54d2eaf81636334b"} Nov 04 03:52:56.680066 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.678305 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr"] Nov 04 03:52:56.682333 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.682304 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.691525 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.691462 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-mb97r\"" Nov 04 03:52:56.696006 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.695936 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Nov 04 03:52:56.696006 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.695936 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Nov 04 03:52:56.696006 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.695976 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Nov 04 03:52:56.696006 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.696005 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr"] Nov 04 03:52:56.774045 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.773982 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa196927-03ec-4015-8172-44098857d262-workload-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.774045 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.774035 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa196927-03ec-4015-8172-44098857d262-istio-token\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.774327 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.774082 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa196927-03ec-4015-8172-44098857d262-workload-certs\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.774327 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.774105 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa196927-03ec-4015-8172-44098857d262-istio-data\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.774327 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.774131 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa196927-03ec-4015-8172-44098857d262-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.774327 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.774159 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa196927-03ec-4015-8172-44098857d262-credential-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.774327 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.774218 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa196927-03ec-4015-8172-44098857d262-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.774327 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.774257 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdzsh\" (UniqueName: \"kubernetes.io/projected/fa196927-03ec-4015-8172-44098857d262-kube-api-access-hdzsh\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.774327 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.774289 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa196927-03ec-4015-8172-44098857d262-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.878702 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.875751 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa196927-03ec-4015-8172-44098857d262-workload-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.878702 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.875821 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa196927-03ec-4015-8172-44098857d262-istio-token\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.878702 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.875870 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa196927-03ec-4015-8172-44098857d262-workload-certs\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.878702 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.875894 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa196927-03ec-4015-8172-44098857d262-istio-data\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.878702 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.875923 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa196927-03ec-4015-8172-44098857d262-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.878702 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.875952 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa196927-03ec-4015-8172-44098857d262-credential-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.878702 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.876018 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa196927-03ec-4015-8172-44098857d262-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.878702 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.876065 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdzsh\" (UniqueName: \"kubernetes.io/projected/fa196927-03ec-4015-8172-44098857d262-kube-api-access-hdzsh\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.878702 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.876099 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa196927-03ec-4015-8172-44098857d262-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.878702 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.876942 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa196927-03ec-4015-8172-44098857d262-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.878702 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.877242 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa196927-03ec-4015-8172-44098857d262-workload-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.878702 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.877830 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa196927-03ec-4015-8172-44098857d262-workload-certs\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.878702 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.878037 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa196927-03ec-4015-8172-44098857d262-istio-data\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.878702 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.878677 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa196927-03ec-4015-8172-44098857d262-credential-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.884581 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.884549 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa196927-03ec-4015-8172-44098857d262-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.884978 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.884893 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa196927-03ec-4015-8172-44098857d262-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.892657 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.892623 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa196927-03ec-4015-8172-44098857d262-istio-token\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.894389 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.894226 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdzsh\" (UniqueName: \"kubernetes.io/projected/fa196927-03ec-4015-8172-44098857d262-kube-api-access-hdzsh\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-skgsr\" (UID: \"fa196927-03ec-4015-8172-44098857d262\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:56.930175 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.928220 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7"] Nov 04 03:52:56.933858 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.933815 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7" Nov 04 03:52:56.940897 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.939807 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-jvtx7\"" Nov 04 03:52:56.941456 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.941400 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7"] Nov 04 03:52:56.977636 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.977348 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a561b20-1967-40af-8d70-dcd6178e51c3-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7\" (UID: \"2a561b20-1967-40af-8d70-dcd6178e51c3\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7" Nov 04 03:52:56.977636 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.977411 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a561b20-1967-40af-8d70-dcd6178e51c3-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7\" (UID: \"2a561b20-1967-40af-8d70-dcd6178e51c3\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7" Nov 04 03:52:56.977636 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:56.977523 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngbzt\" (UniqueName: \"kubernetes.io/projected/2a561b20-1967-40af-8d70-dcd6178e51c3-kube-api-access-ngbzt\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7\" (UID: \"2a561b20-1967-40af-8d70-dcd6178e51c3\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7" Nov 04 03:52:57.004014 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:57.003810 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:52:57.080158 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:57.078724 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngbzt\" (UniqueName: \"kubernetes.io/projected/2a561b20-1967-40af-8d70-dcd6178e51c3-kube-api-access-ngbzt\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7\" (UID: \"2a561b20-1967-40af-8d70-dcd6178e51c3\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7" Nov 04 03:52:57.080158 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:57.078806 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a561b20-1967-40af-8d70-dcd6178e51c3-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7\" (UID: \"2a561b20-1967-40af-8d70-dcd6178e51c3\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7" Nov 04 03:52:57.080158 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:57.078851 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a561b20-1967-40af-8d70-dcd6178e51c3-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7\" (UID: \"2a561b20-1967-40af-8d70-dcd6178e51c3\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7" Nov 04 03:52:57.080158 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:57.079373 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a561b20-1967-40af-8d70-dcd6178e51c3-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7\" (UID: \"2a561b20-1967-40af-8d70-dcd6178e51c3\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7" Nov 04 03:52:57.080158 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:57.080057 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a561b20-1967-40af-8d70-dcd6178e51c3-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7\" (UID: \"2a561b20-1967-40af-8d70-dcd6178e51c3\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7" Nov 04 03:52:57.093004 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:57.092952 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngbzt\" (UniqueName: \"kubernetes.io/projected/2a561b20-1967-40af-8d70-dcd6178e51c3-kube-api-access-ngbzt\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7\" (UID: \"2a561b20-1967-40af-8d70-dcd6178e51c3\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7" Nov 04 03:52:57.258851 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:57.258603 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7" Nov 04 03:52:57.320164 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:57.320126 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr"] Nov 04 03:52:57.578265 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:57.578190 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7"] Nov 04 03:52:57.583049 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:52:57.583020 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a561b20_1967_40af_8d70_dcd6178e51c3.slice/crio-8b95be1a03e1d21f04c5d860037f1f14caaa0c743397ec7abe79f4567ccd399c WatchSource:0}: Error finding container 8b95be1a03e1d21f04c5d860037f1f14caaa0c743397ec7abe79f4567ccd399c: Status 404 returned error can't find the container with id 8b95be1a03e1d21f04c5d860037f1f14caaa0c743397ec7abe79f4567ccd399c Nov 04 03:52:57.805593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:57.805472 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7" event={"ID":"2a561b20-1967-40af-8d70-dcd6178e51c3","Type":"ContainerStarted","Data":"8b95be1a03e1d21f04c5d860037f1f14caaa0c743397ec7abe79f4567ccd399c"} Nov 04 03:52:57.809438 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:52:57.809345 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" event={"ID":"fa196927-03ec-4015-8172-44098857d262","Type":"ContainerStarted","Data":"277878410ca28cbe14be95a575ecd77e63e734479de7efe4d593919becb419bb"} Nov 04 03:53:05.724616 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:05.724571 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:53:05.725138 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:05.724648 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:53:05.725138 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:05.724689 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:53:06.895070 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:06.895024 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" event={"ID":"fa196927-03ec-4015-8172-44098857d262","Type":"ContainerStarted","Data":"b92c0b4524a6f0c422cae13f885342196d0ec821f28607d2855a26f00162651e"} Nov 04 03:53:06.924270 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:06.924199 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" podStartSLOduration=2.5303211 podStartE2EDuration="10.924178115s" totalImagesPullingTime="8.393857015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:52:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:53:06.921313716 +0000 UTC m=+184.959753693" watchObservedRunningTime="2026-11-04 03:53:06.924178115 +0000 UTC m=+184.962618076" Nov 04 03:53:07.005060 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:07.004997 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:53:07.010930 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:07.010751 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:53:07.901330 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:07.901291 6000 generic.go:356] "Generic (PLEG): container finished" podID="639c38c2-be7f-4e49-bc66-8ff5a640b6c5" containerID="cee35ed8844da55e22f9ceb357e66a882a8856ae62997e3d1f6534d9fc42678e" exitCode=0 Nov 04 03:53:07.901719 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:07.901350 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq" event={"ID":"639c38c2-be7f-4e49-bc66-8ff5a640b6c5","Type":"ContainerDied","Data":"cee35ed8844da55e22f9ceb357e66a882a8856ae62997e3d1f6534d9fc42678e"} Nov 04 03:53:07.904237 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:07.903662 6000 generic.go:356] "Generic (PLEG): container finished" podID="eb4f9c7b-e5ae-4aed-af9d-d092cb13add9" containerID="fa3d90291b9e2499a201859fce1d97f1c0f3f59a684b3db6d58eae4103da0eb4" exitCode=0 Nov 04 03:53:07.904237 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:07.903746 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x" event={"ID":"eb4f9c7b-e5ae-4aed-af9d-d092cb13add9","Type":"ContainerDied","Data":"fa3d90291b9e2499a201859fce1d97f1c0f3f59a684b3db6d58eae4103da0eb4"} Nov 04 03:53:07.906274 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:07.906205 6000 generic.go:356] "Generic (PLEG): container finished" podID="b27681f5-ec3b-4fcf-bcaa-4782fb333b88" containerID="c962f4639a633d3302d75da1f0ba56244bd67496e0b5090c4838e2166ae0de83" exitCode=0 Nov 04 03:53:07.906357 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:07.906289 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5" event={"ID":"b27681f5-ec3b-4fcf-bcaa-4782fb333b88","Type":"ContainerDied","Data":"c962f4639a633d3302d75da1f0ba56244bd67496e0b5090c4838e2166ae0de83"} Nov 04 03:53:07.908828 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:07.908807 6000 generic.go:356] "Generic (PLEG): container finished" podID="2a561b20-1967-40af-8d70-dcd6178e51c3" containerID="c771e5aeb6227599c92719ac5653c7464d6e3ba4ceab0f77a366d63ceada67a1" exitCode=0 Nov 04 03:53:07.910072 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:07.909836 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7" event={"ID":"2a561b20-1967-40af-8d70-dcd6178e51c3","Type":"ContainerDied","Data":"c771e5aeb6227599c92719ac5653c7464d6e3ba4ceab0f77a366d63ceada67a1"} Nov 04 03:53:07.910072 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:07.909899 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:53:07.910720 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:07.910699 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:53:08.913632 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:08.913592 6000 generic.go:356] "Generic (PLEG): container finished" podID="c7155f85-0a15-4f1e-aa99-8c51b53f6b61" containerID="b36b8ba8279ea4385dca0b3cd290f683b321b06eff0a22a83bbe4b1f211efe02" exitCode=0 Nov 04 03:53:08.914145 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:08.913685 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc" event={"ID":"c7155f85-0a15-4f1e-aa99-8c51b53f6b61","Type":"ContainerDied","Data":"b36b8ba8279ea4385dca0b3cd290f683b321b06eff0a22a83bbe4b1f211efe02"} Nov 04 03:53:08.915216 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:08.915194 6000 generic.go:356] "Generic (PLEG): container finished" podID="41e4fa82-143e-415e-9549-b859194077ec" containerID="d315cc9cbeaae4ccd560ae3c0f2f3af67d15cd7492f6366bc099c598b5969718" exitCode=0 Nov 04 03:53:08.915314 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:08.915268 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj" event={"ID":"41e4fa82-143e-415e-9549-b859194077ec","Type":"ContainerDied","Data":"d315cc9cbeaae4ccd560ae3c0f2f3af67d15cd7492f6366bc099c598b5969718"} Nov 04 03:53:09.921200 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:09.921157 6000 generic.go:356] "Generic (PLEG): container finished" podID="639c38c2-be7f-4e49-bc66-8ff5a640b6c5" containerID="acafc12a962ab93bfb534a16205707f58b1426a1f5dcdedcc137c837f97adde4" exitCode=0 Nov 04 03:53:09.921639 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:09.921255 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq" event={"ID":"639c38c2-be7f-4e49-bc66-8ff5a640b6c5","Type":"ContainerDied","Data":"acafc12a962ab93bfb534a16205707f58b1426a1f5dcdedcc137c837f97adde4"} Nov 04 03:53:09.923148 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:09.923121 6000 generic.go:356] "Generic (PLEG): container finished" podID="b27681f5-ec3b-4fcf-bcaa-4782fb333b88" containerID="2419bceda4c6d5bed75649df9169342efb16002454b0a17d4ed7f52762b1e519" exitCode=0 Nov 04 03:53:09.923260 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:09.923194 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5" event={"ID":"b27681f5-ec3b-4fcf-bcaa-4782fb333b88","Type":"ContainerDied","Data":"2419bceda4c6d5bed75649df9169342efb16002454b0a17d4ed7f52762b1e519"} Nov 04 03:53:09.926271 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:09.926248 6000 generic.go:356] "Generic (PLEG): container finished" podID="2a561b20-1967-40af-8d70-dcd6178e51c3" containerID="3d0ff7ca77842a3d1471ee8247c58fb18f9d42cacea3dc390540a0ad5bbfc08f" exitCode=0 Nov 04 03:53:09.926382 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:09.926346 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7" event={"ID":"2a561b20-1967-40af-8d70-dcd6178e51c3","Type":"ContainerDied","Data":"3d0ff7ca77842a3d1471ee8247c58fb18f9d42cacea3dc390540a0ad5bbfc08f"} Nov 04 03:53:09.929136 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:09.929067 6000 generic.go:356] "Generic (PLEG): container finished" podID="c070260d-4185-49d8-8467-af038460ca9f" containerID="e0600b09de3ceb1767b99e8fcda132d514d02349a6ed2cf0e7d84a36455de151" exitCode=0 Nov 04 03:53:09.929234 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:09.929142 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7" event={"ID":"c070260d-4185-49d8-8467-af038460ca9f","Type":"ContainerDied","Data":"e0600b09de3ceb1767b99e8fcda132d514d02349a6ed2cf0e7d84a36455de151"} Nov 04 03:53:10.945537 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:10.942578 6000 generic.go:356] "Generic (PLEG): container finished" podID="c7155f85-0a15-4f1e-aa99-8c51b53f6b61" containerID="b8134970b812ddb081b5946260ebb8c80a3207fd287d595954b7a8588adf8003" exitCode=0 Nov 04 03:53:10.945537 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:10.942681 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc" event={"ID":"c7155f85-0a15-4f1e-aa99-8c51b53f6b61","Type":"ContainerDied","Data":"b8134970b812ddb081b5946260ebb8c80a3207fd287d595954b7a8588adf8003"} Nov 04 03:53:10.948891 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:10.948859 6000 generic.go:356] "Generic (PLEG): container finished" podID="41e4fa82-143e-415e-9549-b859194077ec" containerID="be8a0b02a9e5de4f864bb8614dc85e958bc4b1ccf88cfc1b3a79c66ec6cde402" exitCode=0 Nov 04 03:53:10.949030 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:10.948985 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj" event={"ID":"41e4fa82-143e-415e-9549-b859194077ec","Type":"ContainerDied","Data":"be8a0b02a9e5de4f864bb8614dc85e958bc4b1ccf88cfc1b3a79c66ec6cde402"} Nov 04 03:53:11.957509 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:11.957468 6000 generic.go:356] "Generic (PLEG): container finished" podID="eb4f9c7b-e5ae-4aed-af9d-d092cb13add9" containerID="5d91a2faf3e12319aae07dd6bbbd33a3b23cbe7477b0892bb2cea2104b61a0b2" exitCode=0 Nov 04 03:53:11.958022 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:11.957606 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x" event={"ID":"eb4f9c7b-e5ae-4aed-af9d-d092cb13add9","Type":"ContainerDied","Data":"5d91a2faf3e12319aae07dd6bbbd33a3b23cbe7477b0892bb2cea2104b61a0b2"} Nov 04 03:53:15.000159 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:14.999077 6000 generic.go:356] "Generic (PLEG): container finished" podID="c070260d-4185-49d8-8467-af038460ca9f" containerID="147004f1b29bd66f4acc32b71dba2940b8de272684510d7fb11ca802d5ecde8f" exitCode=0 Nov 04 03:53:15.000159 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:14.999185 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7" event={"ID":"c070260d-4185-49d8-8467-af038460ca9f","Type":"ContainerDied","Data":"147004f1b29bd66f4acc32b71dba2940b8de272684510d7fb11ca802d5ecde8f"} Nov 04 03:53:20.768218 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:53:20.768140 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 03:53:25.076774 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:25.076726 6000 generic.go:356] "Generic (PLEG): container finished" podID="639c38c2-be7f-4e49-bc66-8ff5a640b6c5" containerID="079f14a1191d8ec422eab0363987b8686632d5e6f9c50b634b7b22df45e4a614" exitCode=0 Nov 04 03:53:25.077233 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:25.076802 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq" event={"ID":"639c38c2-be7f-4e49-bc66-8ff5a640b6c5","Type":"ContainerDied","Data":"079f14a1191d8ec422eab0363987b8686632d5e6f9c50b634b7b22df45e4a614"} Nov 04 03:53:25.080413 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:25.080388 6000 generic.go:356] "Generic (PLEG): container finished" podID="2a561b20-1967-40af-8d70-dcd6178e51c3" containerID="54e46488508908039b64b143419ffbae6031ff82855be64f70518d1315351b51" exitCode=0 Nov 04 03:53:25.080586 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:25.080568 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7" event={"ID":"2a561b20-1967-40af-8d70-dcd6178e51c3","Type":"ContainerDied","Data":"54e46488508908039b64b143419ffbae6031ff82855be64f70518d1315351b51"} Nov 04 03:53:26.087408 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.087375 6000 generic.go:356] "Generic (PLEG): container finished" podID="c7155f85-0a15-4f1e-aa99-8c51b53f6b61" containerID="fed49728a9b08feb1bae27c6d2977b1f8fb1124eb6a51e0770a6cb5e5aa7de5f" exitCode=0 Nov 04 03:53:26.087894 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.087806 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc" event={"ID":"c7155f85-0a15-4f1e-aa99-8c51b53f6b61","Type":"ContainerDied","Data":"fed49728a9b08feb1bae27c6d2977b1f8fb1124eb6a51e0770a6cb5e5aa7de5f"} Nov 04 03:53:26.090912 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.090876 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj" event={"ID":"41e4fa82-143e-415e-9549-b859194077ec","Type":"ContainerStarted","Data":"e769edd51712ba43a5cf3c114ff8b5d4424ab515f296f193ca35321884962d51"} Nov 04 03:53:26.093396 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.093372 6000 generic.go:356] "Generic (PLEG): container finished" podID="c070260d-4185-49d8-8467-af038460ca9f" containerID="ca7dd3b64345c3e02d6584dfea7eb5cce102c466b6a7fc8ddfe15919494b6d21" exitCode=0 Nov 04 03:53:26.093532 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.093402 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7" event={"ID":"c070260d-4185-49d8-8467-af038460ca9f","Type":"ContainerDied","Data":"ca7dd3b64345c3e02d6584dfea7eb5cce102c466b6a7fc8ddfe15919494b6d21"} Nov 04 03:53:26.095882 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.095860 6000 generic.go:356] "Generic (PLEG): container finished" podID="b27681f5-ec3b-4fcf-bcaa-4782fb333b88" containerID="f0c7617859be86a1cae52d7235e9b301c40678a8a2a9f508cfaef3ec47f7ddb7" exitCode=0 Nov 04 03:53:26.096003 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.095950 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5" event={"ID":"b27681f5-ec3b-4fcf-bcaa-4782fb333b88","Type":"ContainerDied","Data":"f0c7617859be86a1cae52d7235e9b301c40678a8a2a9f508cfaef3ec47f7ddb7"} Nov 04 03:53:26.192253 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.192196 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj" podStartSLOduration=2.869262728 podStartE2EDuration="34.192176066s" totalImagesPullingTime="31.322913338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:52:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:53:26.190476621 +0000 UTC m=+204.228916611" watchObservedRunningTime="2026-11-04 03:53:26.192176066 +0000 UTC m=+204.230616031" Nov 04 03:53:26.267704 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.267628 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7" Nov 04 03:53:26.293772 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.293728 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq" Nov 04 03:53:26.380887 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.380851 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/639c38c2-be7f-4e49-bc66-8ff5a640b6c5-util\" (UniqueName: \"kubernetes.io/empty-dir/639c38c2-be7f-4e49-bc66-8ff5a640b6c5-util\") pod \"639c38c2-be7f-4e49-bc66-8ff5a640b6c5\" (UID: \"639c38c2-be7f-4e49-bc66-8ff5a640b6c5\") " Nov 04 03:53:26.381055 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.380907 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/639c38c2-be7f-4e49-bc66-8ff5a640b6c5-bundle\" (UniqueName: \"kubernetes.io/empty-dir/639c38c2-be7f-4e49-bc66-8ff5a640b6c5-bundle\") pod \"639c38c2-be7f-4e49-bc66-8ff5a640b6c5\" (UID: \"639c38c2-be7f-4e49-bc66-8ff5a640b6c5\") " Nov 04 03:53:26.381055 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.380938 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/639c38c2-be7f-4e49-bc66-8ff5a640b6c5-kube-api-access-7ns56\" (UniqueName: \"kubernetes.io/projected/639c38c2-be7f-4e49-bc66-8ff5a640b6c5-kube-api-access-7ns56\") pod \"639c38c2-be7f-4e49-bc66-8ff5a640b6c5\" (UID: \"639c38c2-be7f-4e49-bc66-8ff5a640b6c5\") " Nov 04 03:53:26.381055 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.380997 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a561b20-1967-40af-8d70-dcd6178e51c3-util\" (UniqueName: \"kubernetes.io/empty-dir/2a561b20-1967-40af-8d70-dcd6178e51c3-util\") pod \"2a561b20-1967-40af-8d70-dcd6178e51c3\" (UID: \"2a561b20-1967-40af-8d70-dcd6178e51c3\") " Nov 04 03:53:26.381055 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.381027 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a561b20-1967-40af-8d70-dcd6178e51c3-kube-api-access-ngbzt\" (UniqueName: \"kubernetes.io/projected/2a561b20-1967-40af-8d70-dcd6178e51c3-kube-api-access-ngbzt\") pod \"2a561b20-1967-40af-8d70-dcd6178e51c3\" (UID: \"2a561b20-1967-40af-8d70-dcd6178e51c3\") " Nov 04 03:53:26.381248 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.381093 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a561b20-1967-40af-8d70-dcd6178e51c3-bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a561b20-1967-40af-8d70-dcd6178e51c3-bundle\") pod \"2a561b20-1967-40af-8d70-dcd6178e51c3\" (UID: \"2a561b20-1967-40af-8d70-dcd6178e51c3\") " Nov 04 03:53:26.381813 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.381774 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a561b20-1967-40af-8d70-dcd6178e51c3-bundle" pod "2a561b20-1967-40af-8d70-dcd6178e51c3" (UID: "2a561b20-1967-40af-8d70-dcd6178e51c3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 03:53:26.381953 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.381874 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/639c38c2-be7f-4e49-bc66-8ff5a640b6c5-bundle" pod "639c38c2-be7f-4e49-bc66-8ff5a640b6c5" (UID: "639c38c2-be7f-4e49-bc66-8ff5a640b6c5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 03:53:26.383432 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.383400 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a561b20-1967-40af-8d70-dcd6178e51c3-kube-api-access-ngbzt" pod "2a561b20-1967-40af-8d70-dcd6178e51c3" (UID: "2a561b20-1967-40af-8d70-dcd6178e51c3"). InnerVolumeSpecName "kube-api-access-ngbzt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 03:53:26.383526 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.383513 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/639c38c2-be7f-4e49-bc66-8ff5a640b6c5-kube-api-access-7ns56" pod "639c38c2-be7f-4e49-bc66-8ff5a640b6c5" (UID: "639c38c2-be7f-4e49-bc66-8ff5a640b6c5"). InnerVolumeSpecName "kube-api-access-7ns56". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 03:53:26.385826 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.385799 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/639c38c2-be7f-4e49-bc66-8ff5a640b6c5-util" pod "639c38c2-be7f-4e49-bc66-8ff5a640b6c5" (UID: "639c38c2-be7f-4e49-bc66-8ff5a640b6c5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 03:53:26.387113 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.387093 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a561b20-1967-40af-8d70-dcd6178e51c3-util" pod "2a561b20-1967-40af-8d70-dcd6178e51c3" (UID: "2a561b20-1967-40af-8d70-dcd6178e51c3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 03:53:26.482673 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.482639 6000 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a561b20-1967-40af-8d70-dcd6178e51c3-bundle\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:53:26.482673 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.482666 6000 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/639c38c2-be7f-4e49-bc66-8ff5a640b6c5-util\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:53:26.482673 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.482675 6000 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/639c38c2-be7f-4e49-bc66-8ff5a640b6c5-bundle\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:53:26.482905 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.482684 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ns56\" (UniqueName: \"kubernetes.io/projected/639c38c2-be7f-4e49-bc66-8ff5a640b6c5-kube-api-access-7ns56\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:53:26.482905 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.482694 6000 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a561b20-1967-40af-8d70-dcd6178e51c3-util\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:53:26.482905 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:26.482704 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngbzt\" (UniqueName: \"kubernetes.io/projected/2a561b20-1967-40af-8d70-dcd6178e51c3-kube-api-access-ngbzt\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:53:27.101290 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.101258 6000 generic.go:356] "Generic (PLEG): container finished" podID="41e4fa82-143e-415e-9549-b859194077ec" containerID="e769edd51712ba43a5cf3c114ff8b5d4424ab515f296f193ca35321884962d51" exitCode=0 Nov 04 03:53:27.101712 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.101349 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj" event={"ID":"41e4fa82-143e-415e-9549-b859194077ec","Type":"ContainerDied","Data":"e769edd51712ba43a5cf3c114ff8b5d4424ab515f296f193ca35321884962d51"} Nov 04 03:53:27.103284 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.103265 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq" Nov 04 03:53:27.103418 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.103264 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2sjsq" event={"ID":"639c38c2-be7f-4e49-bc66-8ff5a640b6c5","Type":"ContainerDied","Data":"ce65697dcd258d22b2fc91267d5ef75caf17fc99d4abbed3a7881d3dcd5e6c89"} Nov 04 03:53:27.103418 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.103394 6000 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce65697dcd258d22b2fc91267d5ef75caf17fc99d4abbed3a7881d3dcd5e6c89" Nov 04 03:53:27.105309 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.105277 6000 generic.go:356] "Generic (PLEG): container finished" podID="eb4f9c7b-e5ae-4aed-af9d-d092cb13add9" containerID="da998c0bccca665dd7b1decd000d0bb4a950564141882775ea868a59a0a4513a" exitCode=0 Nov 04 03:53:27.105445 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.105359 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x" event={"ID":"eb4f9c7b-e5ae-4aed-af9d-d092cb13add9","Type":"ContainerDied","Data":"da998c0bccca665dd7b1decd000d0bb4a950564141882775ea868a59a0a4513a"} Nov 04 03:53:27.107335 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.107310 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7" event={"ID":"2a561b20-1967-40af-8d70-dcd6178e51c3","Type":"ContainerDied","Data":"8b95be1a03e1d21f04c5d860037f1f14caaa0c743397ec7abe79f4567ccd399c"} Nov 04 03:53:27.107455 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.107335 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71snkt7" Nov 04 03:53:27.107455 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.107352 6000 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8b95be1a03e1d21f04c5d860037f1f14caaa0c743397ec7abe79f4567ccd399c" Nov 04 03:53:27.238571 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.238539 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7" Nov 04 03:53:27.268035 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.267995 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc" Nov 04 03:53:27.275831 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.275807 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5" Nov 04 03:53:27.390067 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.389973 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7155f85-0a15-4f1e-aa99-8c51b53f6b61-util\" (UniqueName: \"kubernetes.io/empty-dir/c7155f85-0a15-4f1e-aa99-8c51b53f6b61-util\") pod \"c7155f85-0a15-4f1e-aa99-8c51b53f6b61\" (UID: \"c7155f85-0a15-4f1e-aa99-8c51b53f6b61\") " Nov 04 03:53:27.390067 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.390029 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b27681f5-ec3b-4fcf-bcaa-4782fb333b88-bundle\" (UniqueName: \"kubernetes.io/empty-dir/b27681f5-ec3b-4fcf-bcaa-4782fb333b88-bundle\") pod \"b27681f5-ec3b-4fcf-bcaa-4782fb333b88\" (UID: \"b27681f5-ec3b-4fcf-bcaa-4782fb333b88\") " Nov 04 03:53:27.390067 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.390060 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c070260d-4185-49d8-8467-af038460ca9f-util\" (UniqueName: \"kubernetes.io/empty-dir/c070260d-4185-49d8-8467-af038460ca9f-util\") pod \"c070260d-4185-49d8-8467-af038460ca9f\" (UID: \"c070260d-4185-49d8-8467-af038460ca9f\") " Nov 04 03:53:27.390335 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.390094 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c7155f85-0a15-4f1e-aa99-8c51b53f6b61-kube-api-access-dcvs9\" (UniqueName: \"kubernetes.io/projected/c7155f85-0a15-4f1e-aa99-8c51b53f6b61-kube-api-access-dcvs9\") pod \"c7155f85-0a15-4f1e-aa99-8c51b53f6b61\" (UID: \"c7155f85-0a15-4f1e-aa99-8c51b53f6b61\") " Nov 04 03:53:27.390335 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.390126 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c070260d-4185-49d8-8467-af038460ca9f-kube-api-access-t7h6t\" (UniqueName: \"kubernetes.io/projected/c070260d-4185-49d8-8467-af038460ca9f-kube-api-access-t7h6t\") pod \"c070260d-4185-49d8-8467-af038460ca9f\" (UID: \"c070260d-4185-49d8-8467-af038460ca9f\") " Nov 04 03:53:27.390335 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.390289 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7155f85-0a15-4f1e-aa99-8c51b53f6b61-bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7155f85-0a15-4f1e-aa99-8c51b53f6b61-bundle\") pod \"c7155f85-0a15-4f1e-aa99-8c51b53f6b61\" (UID: \"c7155f85-0a15-4f1e-aa99-8c51b53f6b61\") " Nov 04 03:53:27.390498 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.390351 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c070260d-4185-49d8-8467-af038460ca9f-bundle\" (UniqueName: \"kubernetes.io/empty-dir/c070260d-4185-49d8-8467-af038460ca9f-bundle\") pod \"c070260d-4185-49d8-8467-af038460ca9f\" (UID: \"c070260d-4185-49d8-8467-af038460ca9f\") " Nov 04 03:53:27.390498 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.390395 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b27681f5-ec3b-4fcf-bcaa-4782fb333b88-util\" (UniqueName: \"kubernetes.io/empty-dir/b27681f5-ec3b-4fcf-bcaa-4782fb333b88-util\") pod \"b27681f5-ec3b-4fcf-bcaa-4782fb333b88\" (UID: \"b27681f5-ec3b-4fcf-bcaa-4782fb333b88\") " Nov 04 03:53:27.390498 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.390441 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b27681f5-ec3b-4fcf-bcaa-4782fb333b88-kube-api-access-hnxh9\" (UniqueName: \"kubernetes.io/projected/b27681f5-ec3b-4fcf-bcaa-4782fb333b88-kube-api-access-hnxh9\") pod \"b27681f5-ec3b-4fcf-bcaa-4782fb333b88\" (UID: \"b27681f5-ec3b-4fcf-bcaa-4782fb333b88\") " Nov 04 03:53:27.391066 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.390999 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b27681f5-ec3b-4fcf-bcaa-4782fb333b88-bundle" pod "b27681f5-ec3b-4fcf-bcaa-4782fb333b88" (UID: "b27681f5-ec3b-4fcf-bcaa-4782fb333b88"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 03:53:27.391281 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.391250 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7155f85-0a15-4f1e-aa99-8c51b53f6b61-bundle" pod "c7155f85-0a15-4f1e-aa99-8c51b53f6b61" (UID: "c7155f85-0a15-4f1e-aa99-8c51b53f6b61"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 03:53:27.391367 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.391353 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c070260d-4185-49d8-8467-af038460ca9f-bundle" pod "c070260d-4185-49d8-8467-af038460ca9f" (UID: "c070260d-4185-49d8-8467-af038460ca9f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 03:53:27.392462 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.392445 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c070260d-4185-49d8-8467-af038460ca9f-kube-api-access-t7h6t" pod "c070260d-4185-49d8-8467-af038460ca9f" (UID: "c070260d-4185-49d8-8467-af038460ca9f"). InnerVolumeSpecName "kube-api-access-t7h6t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 03:53:27.392961 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.392932 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b27681f5-ec3b-4fcf-bcaa-4782fb333b88-kube-api-access-hnxh9" pod "b27681f5-ec3b-4fcf-bcaa-4782fb333b88" (UID: "b27681f5-ec3b-4fcf-bcaa-4782fb333b88"). InnerVolumeSpecName "kube-api-access-hnxh9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 03:53:27.394345 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.394325 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7155f85-0a15-4f1e-aa99-8c51b53f6b61-kube-api-access-dcvs9" pod "c7155f85-0a15-4f1e-aa99-8c51b53f6b61" (UID: "c7155f85-0a15-4f1e-aa99-8c51b53f6b61"). InnerVolumeSpecName "kube-api-access-dcvs9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 03:53:27.398832 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.398811 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c070260d-4185-49d8-8467-af038460ca9f-util" pod "c070260d-4185-49d8-8467-af038460ca9f" (UID: "c070260d-4185-49d8-8467-af038460ca9f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 03:53:27.399614 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.399592 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7155f85-0a15-4f1e-aa99-8c51b53f6b61-util" pod "c7155f85-0a15-4f1e-aa99-8c51b53f6b61" (UID: "c7155f85-0a15-4f1e-aa99-8c51b53f6b61"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 03:53:27.399704 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.399686 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b27681f5-ec3b-4fcf-bcaa-4782fb333b88-util" pod "b27681f5-ec3b-4fcf-bcaa-4782fb333b88" (UID: "b27681f5-ec3b-4fcf-bcaa-4782fb333b88"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 03:53:27.491600 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.491557 6000 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7155f85-0a15-4f1e-aa99-8c51b53f6b61-bundle\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:53:27.491600 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.491594 6000 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c070260d-4185-49d8-8467-af038460ca9f-bundle\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:53:27.491600 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.491603 6000 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b27681f5-ec3b-4fcf-bcaa-4782fb333b88-util\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:53:27.491850 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.491611 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnxh9\" (UniqueName: \"kubernetes.io/projected/b27681f5-ec3b-4fcf-bcaa-4782fb333b88-kube-api-access-hnxh9\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:53:27.491850 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.491621 6000 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7155f85-0a15-4f1e-aa99-8c51b53f6b61-util\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:53:27.491850 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.491629 6000 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b27681f5-ec3b-4fcf-bcaa-4782fb333b88-bundle\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:53:27.491850 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.491638 6000 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c070260d-4185-49d8-8467-af038460ca9f-util\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:53:27.491850 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.491647 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcvs9\" (UniqueName: \"kubernetes.io/projected/c7155f85-0a15-4f1e-aa99-8c51b53f6b61-kube-api-access-dcvs9\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:53:27.491850 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:27.491656 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7h6t\" (UniqueName: \"kubernetes.io/projected/c070260d-4185-49d8-8467-af038460ca9f-kube-api-access-t7h6t\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:53:28.113255 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.113229 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5" Nov 04 03:53:28.113255 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.113243 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72npd5" event={"ID":"b27681f5-ec3b-4fcf-bcaa-4782fb333b88","Type":"ContainerDied","Data":"e77a065fb20e9cb0ad443360f76e8577b1bfaa55224bf793050cf3eebd99352b"} Nov 04 03:53:28.113657 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.113288 6000 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e77a065fb20e9cb0ad443360f76e8577b1bfaa55224bf793050cf3eebd99352b" Nov 04 03:53:28.115442 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.115419 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc" event={"ID":"c7155f85-0a15-4f1e-aa99-8c51b53f6b61","Type":"ContainerDied","Data":"545c2758ff09e24cf1dc51a98f01f271586e91b147e9ef543ca58e5aaba99411"} Nov 04 03:53:28.115562 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.115454 6000 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="545c2758ff09e24cf1dc51a98f01f271586e91b147e9ef543ca58e5aaba99411" Nov 04 03:53:28.115615 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.115560 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cft9wc" Nov 04 03:53:28.117534 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.117447 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7" event={"ID":"c070260d-4185-49d8-8467-af038460ca9f","Type":"ContainerDied","Data":"00d913251e6be6f81044985ef22be2894e48c060b27a411b54d2eaf81636334b"} Nov 04 03:53:28.117534 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.117501 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6lgg7" Nov 04 03:53:28.117534 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.117521 6000 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="00d913251e6be6f81044985ef22be2894e48c060b27a411b54d2eaf81636334b" Nov 04 03:53:28.238718 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.238686 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj" Nov 04 03:53:28.272235 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.272207 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x" Nov 04 03:53:28.399712 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.399622 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb4f9c7b-e5ae-4aed-af9d-d092cb13add9-kube-api-access-lrktw\" (UniqueName: \"kubernetes.io/projected/eb4f9c7b-e5ae-4aed-af9d-d092cb13add9-kube-api-access-lrktw\") pod \"eb4f9c7b-e5ae-4aed-af9d-d092cb13add9\" (UID: \"eb4f9c7b-e5ae-4aed-af9d-d092cb13add9\") " Nov 04 03:53:28.399712 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.399693 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41e4fa82-143e-415e-9549-b859194077ec-bundle\" (UniqueName: \"kubernetes.io/empty-dir/41e4fa82-143e-415e-9549-b859194077ec-bundle\") pod \"41e4fa82-143e-415e-9549-b859194077ec\" (UID: \"41e4fa82-143e-415e-9549-b859194077ec\") " Nov 04 03:53:28.399712 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.399713 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb4f9c7b-e5ae-4aed-af9d-d092cb13add9-bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9c7b-e5ae-4aed-af9d-d092cb13add9-bundle\") pod \"eb4f9c7b-e5ae-4aed-af9d-d092cb13add9\" (UID: \"eb4f9c7b-e5ae-4aed-af9d-d092cb13add9\") " Nov 04 03:53:28.400014 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.399737 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/41e4fa82-143e-415e-9549-b859194077ec-kube-api-access-lztkk\" (UniqueName: \"kubernetes.io/projected/41e4fa82-143e-415e-9549-b859194077ec-kube-api-access-lztkk\") pod \"41e4fa82-143e-415e-9549-b859194077ec\" (UID: \"41e4fa82-143e-415e-9549-b859194077ec\") " Nov 04 03:53:28.400014 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.399780 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41e4fa82-143e-415e-9549-b859194077ec-util\" (UniqueName: \"kubernetes.io/empty-dir/41e4fa82-143e-415e-9549-b859194077ec-util\") pod \"41e4fa82-143e-415e-9549-b859194077ec\" (UID: \"41e4fa82-143e-415e-9549-b859194077ec\") " Nov 04 03:53:28.400014 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.399802 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb4f9c7b-e5ae-4aed-af9d-d092cb13add9-util\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9c7b-e5ae-4aed-af9d-d092cb13add9-util\") pod \"eb4f9c7b-e5ae-4aed-af9d-d092cb13add9\" (UID: \"eb4f9c7b-e5ae-4aed-af9d-d092cb13add9\") " Nov 04 03:53:28.400218 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.400188 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb4f9c7b-e5ae-4aed-af9d-d092cb13add9-bundle" pod "eb4f9c7b-e5ae-4aed-af9d-d092cb13add9" (UID: "eb4f9c7b-e5ae-4aed-af9d-d092cb13add9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 03:53:28.400340 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.400289 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41e4fa82-143e-415e-9549-b859194077ec-bundle" pod "41e4fa82-143e-415e-9549-b859194077ec" (UID: "41e4fa82-143e-415e-9549-b859194077ec"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 03:53:28.402035 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.402013 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41e4fa82-143e-415e-9549-b859194077ec-kube-api-access-lztkk" pod "41e4fa82-143e-415e-9549-b859194077ec" (UID: "41e4fa82-143e-415e-9549-b859194077ec"). InnerVolumeSpecName "kube-api-access-lztkk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 03:53:28.402219 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.402199 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb4f9c7b-e5ae-4aed-af9d-d092cb13add9-kube-api-access-lrktw" pod "eb4f9c7b-e5ae-4aed-af9d-d092cb13add9" (UID: "eb4f9c7b-e5ae-4aed-af9d-d092cb13add9"). InnerVolumeSpecName "kube-api-access-lrktw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 03:53:28.405940 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.405916 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb4f9c7b-e5ae-4aed-af9d-d092cb13add9-util" pod "eb4f9c7b-e5ae-4aed-af9d-d092cb13add9" (UID: "eb4f9c7b-e5ae-4aed-af9d-d092cb13add9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 03:53:28.406275 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.406255 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41e4fa82-143e-415e-9549-b859194077ec-util" pod "41e4fa82-143e-415e-9549-b859194077ec" (UID: "41e4fa82-143e-415e-9549-b859194077ec"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 03:53:28.500664 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.500626 6000 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9c7b-e5ae-4aed-af9d-d092cb13add9-bundle\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:53:28.500664 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.500658 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lztkk\" (UniqueName: \"kubernetes.io/projected/41e4fa82-143e-415e-9549-b859194077ec-kube-api-access-lztkk\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:53:28.500664 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.500669 6000 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41e4fa82-143e-415e-9549-b859194077ec-util\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:53:28.500920 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.500677 6000 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9c7b-e5ae-4aed-af9d-d092cb13add9-util\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:53:28.500920 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.500686 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrktw\" (UniqueName: \"kubernetes.io/projected/eb4f9c7b-e5ae-4aed-af9d-d092cb13add9-kube-api-access-lrktw\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:53:28.500920 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:28.500694 6000 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41e4fa82-143e-415e-9549-b859194077ec-bundle\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:53:29.127981 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:29.127882 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x" event={"ID":"eb4f9c7b-e5ae-4aed-af9d-d092cb13add9","Type":"ContainerDied","Data":"a4187619610df9759e5cd6bd05cc252e901b79f751695a75d8d0484b35b9f1c6"} Nov 04 03:53:29.127981 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:29.127971 6000 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a4187619610df9759e5cd6bd05cc252e901b79f751695a75d8d0484b35b9f1c6" Nov 04 03:53:29.128435 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:29.127893 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644mdq9x" Nov 04 03:53:29.129833 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:29.129794 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj" event={"ID":"41e4fa82-143e-415e-9549-b859194077ec","Type":"ContainerDied","Data":"a8f3940070e4e385fd3f03612e1085e0ea44ce269206470f4ffd0e388ff04320"} Nov 04 03:53:29.129940 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:29.129840 6000 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a8f3940070e4e385fd3f03612e1085e0ea44ce269206470f4ffd0e388ff04320" Nov 04 03:53:29.129940 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:29.129842 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62cvzj" Nov 04 03:53:30.978306 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978270 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-vp5lg"] Nov 04 03:53:30.978695 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978571 6000 cpu_manager.go:414] "containerMap: removing container" podUID="2a561b20-1967-40af-8d70-dcd6178e51c3" containerName="util" Nov 04 03:53:30.978695 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978583 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a561b20-1967-40af-8d70-dcd6178e51c3" containerName="util" Nov 04 03:53:30.978695 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978591 6000 cpu_manager.go:414] "containerMap: removing container" podUID="41e4fa82-143e-415e-9549-b859194077ec" containerName="util" Nov 04 03:53:30.978695 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978596 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="41e4fa82-143e-415e-9549-b859194077ec" containerName="util" Nov 04 03:53:30.978695 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978604 6000 cpu_manager.go:414] "containerMap: removing container" podUID="c7155f85-0a15-4f1e-aa99-8c51b53f6b61" containerName="pull" Nov 04 03:53:30.978695 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978610 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c7155f85-0a15-4f1e-aa99-8c51b53f6b61" containerName="pull" Nov 04 03:53:30.978695 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978616 6000 cpu_manager.go:414] "containerMap: removing container" podUID="c7155f85-0a15-4f1e-aa99-8c51b53f6b61" containerName="extract" Nov 04 03:53:30.978695 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978621 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c7155f85-0a15-4f1e-aa99-8c51b53f6b61" containerName="extract" Nov 04 03:53:30.978695 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978631 6000 cpu_manager.go:414] "containerMap: removing container" podUID="41e4fa82-143e-415e-9549-b859194077ec" containerName="pull" Nov 04 03:53:30.978695 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978636 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="41e4fa82-143e-415e-9549-b859194077ec" containerName="pull" Nov 04 03:53:30.978695 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978645 6000 cpu_manager.go:414] "containerMap: removing container" podUID="eb4f9c7b-e5ae-4aed-af9d-d092cb13add9" containerName="extract" Nov 04 03:53:30.978695 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978650 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb4f9c7b-e5ae-4aed-af9d-d092cb13add9" containerName="extract" Nov 04 03:53:30.978695 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978661 6000 cpu_manager.go:414] "containerMap: removing container" podUID="b27681f5-ec3b-4fcf-bcaa-4782fb333b88" containerName="pull" Nov 04 03:53:30.978695 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978665 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b27681f5-ec3b-4fcf-bcaa-4782fb333b88" containerName="pull" Nov 04 03:53:30.978695 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978680 6000 cpu_manager.go:414] "containerMap: removing container" podUID="639c38c2-be7f-4e49-bc66-8ff5a640b6c5" containerName="pull" Nov 04 03:53:30.978695 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978701 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="639c38c2-be7f-4e49-bc66-8ff5a640b6c5" containerName="pull" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978708 6000 cpu_manager.go:414] "containerMap: removing container" podUID="c070260d-4185-49d8-8467-af038460ca9f" containerName="pull" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978712 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c070260d-4185-49d8-8467-af038460ca9f" containerName="pull" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978720 6000 cpu_manager.go:414] "containerMap: removing container" podUID="2a561b20-1967-40af-8d70-dcd6178e51c3" containerName="extract" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978724 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a561b20-1967-40af-8d70-dcd6178e51c3" containerName="extract" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978731 6000 cpu_manager.go:414] "containerMap: removing container" podUID="639c38c2-be7f-4e49-bc66-8ff5a640b6c5" containerName="extract" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978735 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="639c38c2-be7f-4e49-bc66-8ff5a640b6c5" containerName="extract" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978744 6000 cpu_manager.go:414] "containerMap: removing container" podUID="eb4f9c7b-e5ae-4aed-af9d-d092cb13add9" containerName="util" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978749 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb4f9c7b-e5ae-4aed-af9d-d092cb13add9" containerName="util" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978770 6000 cpu_manager.go:414] "containerMap: removing container" podUID="41e4fa82-143e-415e-9549-b859194077ec" containerName="extract" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978775 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="41e4fa82-143e-415e-9549-b859194077ec" containerName="extract" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978784 6000 cpu_manager.go:414] "containerMap: removing container" podUID="639c38c2-be7f-4e49-bc66-8ff5a640b6c5" containerName="util" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978788 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="639c38c2-be7f-4e49-bc66-8ff5a640b6c5" containerName="util" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978794 6000 cpu_manager.go:414] "containerMap: removing container" podUID="c7155f85-0a15-4f1e-aa99-8c51b53f6b61" containerName="util" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978798 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c7155f85-0a15-4f1e-aa99-8c51b53f6b61" containerName="util" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978803 6000 cpu_manager.go:414] "containerMap: removing container" podUID="2a561b20-1967-40af-8d70-dcd6178e51c3" containerName="pull" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978806 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a561b20-1967-40af-8d70-dcd6178e51c3" containerName="pull" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978812 6000 cpu_manager.go:414] "containerMap: removing container" podUID="c070260d-4185-49d8-8467-af038460ca9f" containerName="extract" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978816 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c070260d-4185-49d8-8467-af038460ca9f" containerName="extract" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978821 6000 cpu_manager.go:414] "containerMap: removing container" podUID="b27681f5-ec3b-4fcf-bcaa-4782fb333b88" containerName="extract" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978825 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b27681f5-ec3b-4fcf-bcaa-4782fb333b88" containerName="extract" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978830 6000 cpu_manager.go:414] "containerMap: removing container" podUID="b27681f5-ec3b-4fcf-bcaa-4782fb333b88" containerName="util" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978834 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b27681f5-ec3b-4fcf-bcaa-4782fb333b88" containerName="util" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978840 6000 cpu_manager.go:414] "containerMap: removing container" podUID="c070260d-4185-49d8-8467-af038460ca9f" containerName="util" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978844 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c070260d-4185-49d8-8467-af038460ca9f" containerName="util" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978850 6000 cpu_manager.go:414] "containerMap: removing container" podUID="eb4f9c7b-e5ae-4aed-af9d-d092cb13add9" containerName="pull" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978855 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb4f9c7b-e5ae-4aed-af9d-d092cb13add9" containerName="pull" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978898 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="2a561b20-1967-40af-8d70-dcd6178e51c3" containerName="extract" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978905 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="639c38c2-be7f-4e49-bc66-8ff5a640b6c5" containerName="extract" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978911 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="41e4fa82-143e-415e-9549-b859194077ec" containerName="extract" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978918 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="c070260d-4185-49d8-8467-af038460ca9f" containerName="extract" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978922 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="b27681f5-ec3b-4fcf-bcaa-4782fb333b88" containerName="extract" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978927 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="c7155f85-0a15-4f1e-aa99-8c51b53f6b61" containerName="extract" Nov 04 03:53:30.979172 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.978933 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb4f9c7b-e5ae-4aed-af9d-d092cb13add9" containerName="extract" Nov 04 03:53:30.981113 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.981097 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-vp5lg" Nov 04 03:53:30.990097 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.990066 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-hnm2m\"" Nov 04 03:53:30.990228 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.990122 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Nov 04 03:53:30.990228 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.990148 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Nov 04 03:53:30.990228 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.990201 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Nov 04 03:53:30.995145 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:30.995122 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-vp5lg"] Nov 04 03:53:31.120429 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:31.120386 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnzng\" (UniqueName: \"kubernetes.io/projected/f59b063c-f397-4370-ac72-84335c8115b4-kube-api-access-gnzng\") pod \"dns-operator-controller-manager-8557cbdb84-vp5lg\" (UID: \"f59b063c-f397-4370-ac72-84335c8115b4\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-vp5lg" Nov 04 03:53:31.221778 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:31.221717 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnzng\" (UniqueName: \"kubernetes.io/projected/f59b063c-f397-4370-ac72-84335c8115b4-kube-api-access-gnzng\") pod \"dns-operator-controller-manager-8557cbdb84-vp5lg\" (UID: \"f59b063c-f397-4370-ac72-84335c8115b4\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-vp5lg" Nov 04 03:53:31.235625 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:31.235550 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnzng\" (UniqueName: \"kubernetes.io/projected/f59b063c-f397-4370-ac72-84335c8115b4-kube-api-access-gnzng\") pod \"dns-operator-controller-manager-8557cbdb84-vp5lg\" (UID: \"f59b063c-f397-4370-ac72-84335c8115b4\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-vp5lg" Nov 04 03:53:31.291975 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:31.291935 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-vp5lg" Nov 04 03:53:31.566381 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:31.566199 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-vp5lg"] Nov 04 03:53:31.568978 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:53:31.568952 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf59b063c_f397_4370_ac72_84335c8115b4.slice/crio-c0d456b78a895b134a056bf7ebdf88cfff5102d66aad339c1298567693f61145 WatchSource:0}: Error finding container c0d456b78a895b134a056bf7ebdf88cfff5102d66aad339c1298567693f61145: Status 404 returned error can't find the container with id c0d456b78a895b134a056bf7ebdf88cfff5102d66aad339c1298567693f61145 Nov 04 03:53:32.143170 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:32.143135 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-vp5lg" event={"ID":"f59b063c-f397-4370-ac72-84335c8115b4","Type":"ContainerStarted","Data":"c0d456b78a895b134a056bf7ebdf88cfff5102d66aad339c1298567693f61145"} Nov 04 03:53:34.170398 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:34.170355 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-p9sng"] Nov 04 03:53:34.172715 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:34.172694 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-p9sng" Nov 04 03:53:34.183294 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:34.183269 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-pv7k8\"" Nov 04 03:53:34.184784 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:34.184727 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-p9sng"] Nov 04 03:53:34.349256 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:34.349205 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvg52\" (UniqueName: \"kubernetes.io/projected/e84f739c-2745-44c9-ab3c-d6516ab6855a-kube-api-access-zvg52\") pod \"limitador-operator-controller-manager-5778bb4449-p9sng\" (UID: \"e84f739c-2745-44c9-ab3c-d6516ab6855a\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-p9sng" Nov 04 03:53:34.450400 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:34.450300 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvg52\" (UniqueName: \"kubernetes.io/projected/e84f739c-2745-44c9-ab3c-d6516ab6855a-kube-api-access-zvg52\") pod \"limitador-operator-controller-manager-5778bb4449-p9sng\" (UID: \"e84f739c-2745-44c9-ab3c-d6516ab6855a\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-p9sng" Nov 04 03:53:34.469565 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:34.469535 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvg52\" (UniqueName: \"kubernetes.io/projected/e84f739c-2745-44c9-ab3c-d6516ab6855a-kube-api-access-zvg52\") pod \"limitador-operator-controller-manager-5778bb4449-p9sng\" (UID: \"e84f739c-2745-44c9-ab3c-d6516ab6855a\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-p9sng" Nov 04 03:53:34.487295 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:34.487263 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-p9sng" Nov 04 03:53:34.807969 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:34.807934 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-p9sng"] Nov 04 03:53:34.812425 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:53:34.812388 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode84f739c_2745_44c9_ab3c_d6516ab6855a.slice/crio-6d498486ce2eca4112edd996d59451d0c0659ffa97a50356528278403408b043 WatchSource:0}: Error finding container 6d498486ce2eca4112edd996d59451d0c0659ffa97a50356528278403408b043: Status 404 returned error can't find the container with id 6d498486ce2eca4112edd996d59451d0c0659ffa97a50356528278403408b043 Nov 04 03:53:35.158817 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.158718 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-p9sng" event={"ID":"e84f739c-2745-44c9-ab3c-d6516ab6855a","Type":"ContainerStarted","Data":"6d498486ce2eca4112edd996d59451d0c0659ffa97a50356528278403408b043"} Nov 04 03:53:35.698611 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.698577 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz"] Nov 04 03:53:35.700905 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.700887 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" Nov 04 03:53:35.759550 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.759522 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz"] Nov 04 03:53:35.763091 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.763064 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\"" Nov 04 03:53:35.763350 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.763337 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\"" Nov 04 03:53:35.763769 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.763733 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\"" Nov 04 03:53:35.764092 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.764073 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\"" Nov 04 03:53:35.764299 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.764281 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-9csf6\"" Nov 04 03:53:35.764358 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.764301 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\"" Nov 04 03:53:35.864353 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.864306 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4493817e-586c-4ff4-ba4c-546b488b8925-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74f9c8978b-jtnnz\" (UID: \"4493817e-586c-4ff4-ba4c-546b488b8925\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" Nov 04 03:53:35.864707 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.864663 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4493817e-586c-4ff4-ba4c-546b488b8925-tmp\") pod \"opendatahub-operator-controller-manager-74f9c8978b-jtnnz\" (UID: \"4493817e-586c-4ff4-ba4c-546b488b8925\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" Nov 04 03:53:35.864840 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.864710 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bqll\" (UniqueName: \"kubernetes.io/projected/4493817e-586c-4ff4-ba4c-546b488b8925-kube-api-access-8bqll\") pod \"opendatahub-operator-controller-manager-74f9c8978b-jtnnz\" (UID: \"4493817e-586c-4ff4-ba4c-546b488b8925\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" Nov 04 03:53:35.864920 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.864843 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4493817e-586c-4ff4-ba4c-546b488b8925-metrics-cert\") pod \"opendatahub-operator-controller-manager-74f9c8978b-jtnnz\" (UID: \"4493817e-586c-4ff4-ba4c-546b488b8925\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" Nov 04 03:53:35.864987 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.864927 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4493817e-586c-4ff4-ba4c-546b488b8925-manifests\") pod \"opendatahub-operator-controller-manager-74f9c8978b-jtnnz\" (UID: \"4493817e-586c-4ff4-ba4c-546b488b8925\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" Nov 04 03:53:35.864987 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.864972 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4493817e-586c-4ff4-ba4c-546b488b8925-webhook-cert\") pod \"opendatahub-operator-controller-manager-74f9c8978b-jtnnz\" (UID: \"4493817e-586c-4ff4-ba4c-546b488b8925\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" Nov 04 03:53:35.965701 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.965601 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4493817e-586c-4ff4-ba4c-546b488b8925-webhook-cert\") pod \"opendatahub-operator-controller-manager-74f9c8978b-jtnnz\" (UID: \"4493817e-586c-4ff4-ba4c-546b488b8925\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" Nov 04 03:53:35.965701 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.965691 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4493817e-586c-4ff4-ba4c-546b488b8925-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74f9c8978b-jtnnz\" (UID: \"4493817e-586c-4ff4-ba4c-546b488b8925\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" Nov 04 03:53:35.965981 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.965742 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4493817e-586c-4ff4-ba4c-546b488b8925-tmp\") pod \"opendatahub-operator-controller-manager-74f9c8978b-jtnnz\" (UID: \"4493817e-586c-4ff4-ba4c-546b488b8925\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" Nov 04 03:53:35.965981 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.965790 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bqll\" (UniqueName: \"kubernetes.io/projected/4493817e-586c-4ff4-ba4c-546b488b8925-kube-api-access-8bqll\") pod \"opendatahub-operator-controller-manager-74f9c8978b-jtnnz\" (UID: \"4493817e-586c-4ff4-ba4c-546b488b8925\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" Nov 04 03:53:35.965981 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.965834 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4493817e-586c-4ff4-ba4c-546b488b8925-metrics-cert\") pod \"opendatahub-operator-controller-manager-74f9c8978b-jtnnz\" (UID: \"4493817e-586c-4ff4-ba4c-546b488b8925\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" Nov 04 03:53:35.965981 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.965867 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4493817e-586c-4ff4-ba4c-546b488b8925-manifests\") pod \"opendatahub-operator-controller-manager-74f9c8978b-jtnnz\" (UID: \"4493817e-586c-4ff4-ba4c-546b488b8925\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" Nov 04 03:53:35.966286 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.966255 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4493817e-586c-4ff4-ba4c-546b488b8925-manifests\") pod \"opendatahub-operator-controller-manager-74f9c8978b-jtnnz\" (UID: \"4493817e-586c-4ff4-ba4c-546b488b8925\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" Nov 04 03:53:35.968094 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.968066 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4493817e-586c-4ff4-ba4c-546b488b8925-tmp\") pod \"opendatahub-operator-controller-manager-74f9c8978b-jtnnz\" (UID: \"4493817e-586c-4ff4-ba4c-546b488b8925\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" Nov 04 03:53:35.968426 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.968404 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4493817e-586c-4ff4-ba4c-546b488b8925-metrics-cert\") pod \"opendatahub-operator-controller-manager-74f9c8978b-jtnnz\" (UID: \"4493817e-586c-4ff4-ba4c-546b488b8925\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" Nov 04 03:53:35.968473 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.968404 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4493817e-586c-4ff4-ba4c-546b488b8925-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74f9c8978b-jtnnz\" (UID: \"4493817e-586c-4ff4-ba4c-546b488b8925\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" Nov 04 03:53:35.968473 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.968443 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4493817e-586c-4ff4-ba4c-546b488b8925-webhook-cert\") pod \"opendatahub-operator-controller-manager-74f9c8978b-jtnnz\" (UID: \"4493817e-586c-4ff4-ba4c-546b488b8925\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" Nov 04 03:53:35.978678 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:35.978653 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bqll\" (UniqueName: \"kubernetes.io/projected/4493817e-586c-4ff4-ba4c-546b488b8925-kube-api-access-8bqll\") pod \"opendatahub-operator-controller-manager-74f9c8978b-jtnnz\" (UID: \"4493817e-586c-4ff4-ba4c-546b488b8925\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" Nov 04 03:53:36.013888 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:36.013848 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" Nov 04 03:53:36.164622 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:36.164583 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-vp5lg" event={"ID":"f59b063c-f397-4370-ac72-84335c8115b4","Type":"ContainerStarted","Data":"dee62f19cb29307f84e6cf050cf30b58166ee4b4e4af4bf618bb3174bc7e5e5c"} Nov 04 03:53:36.164846 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:36.164660 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-vp5lg" Nov 04 03:53:36.189137 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:36.189076 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-vp5lg" podStartSLOduration=2.443737278 podStartE2EDuration="6.18905637s" totalImagesPullingTime="3.745319092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:53:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:53:36.1882348 +0000 UTC m=+214.226674765" watchObservedRunningTime="2026-11-04 03:53:36.18905637 +0000 UTC m=+214.227496333" Nov 04 03:53:36.310861 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:36.310834 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz"] Nov 04 03:53:36.313411 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:53:36.313382 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4493817e_586c_4ff4_ba4c_546b488b8925.slice/crio-d686ecc850cd44ccf804b712765f01cde9e6fb753689e691f63f6094abd0e878 WatchSource:0}: Error finding container d686ecc850cd44ccf804b712765f01cde9e6fb753689e691f63f6094abd0e878: Status 404 returned error can't find the container with id d686ecc850cd44ccf804b712765f01cde9e6fb753689e691f63f6094abd0e878 Nov 04 03:53:37.184470 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:37.184426 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" event={"ID":"4493817e-586c-4ff4-ba4c-546b488b8925","Type":"ContainerStarted","Data":"d686ecc850cd44ccf804b712765f01cde9e6fb753689e691f63f6094abd0e878"} Nov 04 03:53:38.191402 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:38.191359 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-p9sng" event={"ID":"e84f739c-2745-44c9-ab3c-d6516ab6855a","Type":"ContainerStarted","Data":"601f450fcb7a7a5619b2050dc0d80cd97af89d3b14746475230bfe0183670dbf"} Nov 04 03:53:38.191865 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:38.191506 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-p9sng" Nov 04 03:53:38.214260 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:38.214192 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-p9sng" podStartSLOduration=1.545780561 podStartE2EDuration="4.214169519s" totalImagesPullingTime="2.668388958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:53:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:53:38.211207864 +0000 UTC m=+216.249647828" watchObservedRunningTime="2026-11-04 03:53:38.214169519 +0000 UTC m=+216.252609485" Nov 04 03:53:41.206085 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:41.206045 6000 generic.go:356] "Generic (PLEG): container finished" podID="4493817e-586c-4ff4-ba4c-546b488b8925" containerID="79c935935bb0837f0ef301b04717f226b4510db312fcfc1aec8d47387ef46e26" exitCode=0 Nov 04 03:53:41.206560 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:41.206095 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" event={"ID":"4493817e-586c-4ff4-ba4c-546b488b8925","Type":"ContainerDied","Data":"79c935935bb0837f0ef301b04717f226b4510db312fcfc1aec8d47387ef46e26"} Nov 04 03:53:42.211980 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:42.211919 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" event={"ID":"4493817e-586c-4ff4-ba4c-546b488b8925","Type":"ContainerStarted","Data":"8a9dc215aebe21ce19b80cdfaae0b2d9b4b5c89daefbb4521d3c2c7f391c4a0a"} Nov 04 03:53:42.212384 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:42.212006 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" Nov 04 03:53:42.283233 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:42.283169 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" podStartSLOduration=2.469491612 podStartE2EDuration="7.283150112s" totalImagesPullingTime="4.367710652s" totalInitContainerRuntime="445.947848ms" isStatefulPod=false podCreationTimestamp="2026-11-04 03:53:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:53:42.279812031 +0000 UTC m=+220.318251995" watchObservedRunningTime="2026-11-04 03:53:42.283150112 +0000 UTC m=+220.321590076" Nov 04 03:53:42.787128 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:42.787092 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-jrjlz"] Nov 04 03:53:42.790461 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:42.790439 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-jrjlz" Nov 04 03:53:42.797736 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:42.797716 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-5m5fk\"" Nov 04 03:53:42.805726 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:42.805700 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-jrjlz"] Nov 04 03:53:42.824958 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:42.824930 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmh9c\" (UniqueName: \"kubernetes.io/projected/ddc1e44a-faa9-4022-a73b-4e1f87ee1210-kube-api-access-wmh9c\") pod \"kuadrant-operator-controller-manager-cf8469d7b-jrjlz\" (UID: \"ddc1e44a-faa9-4022-a73b-4e1f87ee1210\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-jrjlz" Nov 04 03:53:42.825082 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:42.824979 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ddc1e44a-faa9-4022-a73b-4e1f87ee1210-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-jrjlz\" (UID: \"ddc1e44a-faa9-4022-a73b-4e1f87ee1210\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-jrjlz" Nov 04 03:53:42.925865 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:42.925808 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ddc1e44a-faa9-4022-a73b-4e1f87ee1210-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-jrjlz\" (UID: \"ddc1e44a-faa9-4022-a73b-4e1f87ee1210\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-jrjlz" Nov 04 03:53:42.926027 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:42.925924 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmh9c\" (UniqueName: \"kubernetes.io/projected/ddc1e44a-faa9-4022-a73b-4e1f87ee1210-kube-api-access-wmh9c\") pod \"kuadrant-operator-controller-manager-cf8469d7b-jrjlz\" (UID: \"ddc1e44a-faa9-4022-a73b-4e1f87ee1210\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-jrjlz" Nov 04 03:53:42.926212 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:42.926189 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ddc1e44a-faa9-4022-a73b-4e1f87ee1210-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-jrjlz\" (UID: \"ddc1e44a-faa9-4022-a73b-4e1f87ee1210\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-jrjlz" Nov 04 03:53:42.941864 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:42.941834 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmh9c\" (UniqueName: \"kubernetes.io/projected/ddc1e44a-faa9-4022-a73b-4e1f87ee1210-kube-api-access-wmh9c\") pod \"kuadrant-operator-controller-manager-cf8469d7b-jrjlz\" (UID: \"ddc1e44a-faa9-4022-a73b-4e1f87ee1210\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-jrjlz" Nov 04 03:53:43.103361 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:43.103316 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-jrjlz" Nov 04 03:53:43.383929 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:43.383904 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-jrjlz"] Nov 04 03:53:43.386248 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:53:43.386220 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddc1e44a_faa9_4022_a73b_4e1f87ee1210.slice/crio-eec3b0a2f08a2a42098328b9e87cb151c91d85aca8dad17668d53b521a41a1b1 WatchSource:0}: Error finding container eec3b0a2f08a2a42098328b9e87cb151c91d85aca8dad17668d53b521a41a1b1: Status 404 returned error can't find the container with id eec3b0a2f08a2a42098328b9e87cb151c91d85aca8dad17668d53b521a41a1b1 Nov 04 03:53:44.221122 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:44.221085 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-jrjlz" event={"ID":"ddc1e44a-faa9-4022-a73b-4e1f87ee1210","Type":"ContainerStarted","Data":"eec3b0a2f08a2a42098328b9e87cb151c91d85aca8dad17668d53b521a41a1b1"} Nov 04 03:53:47.188573 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:47.187807 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-vp5lg" Nov 04 03:53:48.028533 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:48.028501 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-2d8lw"] Nov 04 03:53:48.035124 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:48.035101 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-2d8lw" Nov 04 03:53:48.055373 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:48.055337 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-zlggc\"" Nov 04 03:53:48.057780 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:48.057736 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-2d8lw"] Nov 04 03:53:48.071984 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:48.071950 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59k46\" (UniqueName: \"kubernetes.io/projected/b279d26f-2d92-4082-8edb-d1c2a26022aa-kube-api-access-59k46\") pod \"authorino-operator-7b7bb4bc77-2d8lw\" (UID: \"b279d26f-2d92-4082-8edb-d1c2a26022aa\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-2d8lw" Nov 04 03:53:48.175425 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:48.175221 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59k46\" (UniqueName: \"kubernetes.io/projected/b279d26f-2d92-4082-8edb-d1c2a26022aa-kube-api-access-59k46\") pod \"authorino-operator-7b7bb4bc77-2d8lw\" (UID: \"b279d26f-2d92-4082-8edb-d1c2a26022aa\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-2d8lw" Nov 04 03:53:48.192632 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:48.192595 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59k46\" (UniqueName: \"kubernetes.io/projected/b279d26f-2d92-4082-8edb-d1c2a26022aa-kube-api-access-59k46\") pod \"authorino-operator-7b7bb4bc77-2d8lw\" (UID: \"b279d26f-2d92-4082-8edb-d1c2a26022aa\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-2d8lw" Nov 04 03:53:48.351209 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:48.351169 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-2d8lw" Nov 04 03:53:48.645786 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:48.645742 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-2d8lw"] Nov 04 03:53:48.648296 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:53:48.648267 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb279d26f_2d92_4082_8edb_d1c2a26022aa.slice/crio-95e3cdcf5568dda09a853c3f5893b7e0e78d3a2caf2bb7e49e94940db10b9883 WatchSource:0}: Error finding container 95e3cdcf5568dda09a853c3f5893b7e0e78d3a2caf2bb7e49e94940db10b9883: Status 404 returned error can't find the container with id 95e3cdcf5568dda09a853c3f5893b7e0e78d3a2caf2bb7e49e94940db10b9883 Nov 04 03:53:49.198749 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:49.198714 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-p9sng" Nov 04 03:53:49.245546 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:49.245506 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-2d8lw" event={"ID":"b279d26f-2d92-4082-8edb-d1c2a26022aa","Type":"ContainerStarted","Data":"95e3cdcf5568dda09a853c3f5893b7e0e78d3a2caf2bb7e49e94940db10b9883"} Nov 04 03:53:50.750118 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:53:50.750073 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 03:53:51.004940 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:51.004850 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6hrfs"] Nov 04 03:53:51.008466 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:51.008442 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6hrfs" Nov 04 03:53:51.016180 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:51.016044 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Nov 04 03:53:51.016180 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:51.016151 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Nov 04 03:53:51.017633 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:51.017606 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-6hz2x\"" Nov 04 03:53:51.025879 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:51.025843 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6hrfs"] Nov 04 03:53:51.101082 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:51.101050 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59qbp\" (UniqueName: \"kubernetes.io/projected/ca79d6cb-be65-4e2f-af47-752ffac53ce1-kube-api-access-59qbp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-6hrfs\" (UID: \"ca79d6cb-be65-4e2f-af47-752ffac53ce1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6hrfs" Nov 04 03:53:51.101212 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:51.101178 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca79d6cb-be65-4e2f-af47-752ffac53ce1-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-6hrfs\" (UID: \"ca79d6cb-be65-4e2f-af47-752ffac53ce1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6hrfs" Nov 04 03:53:51.201943 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:51.201899 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59qbp\" (UniqueName: \"kubernetes.io/projected/ca79d6cb-be65-4e2f-af47-752ffac53ce1-kube-api-access-59qbp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-6hrfs\" (UID: \"ca79d6cb-be65-4e2f-af47-752ffac53ce1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6hrfs" Nov 04 03:53:51.202102 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:51.201968 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca79d6cb-be65-4e2f-af47-752ffac53ce1-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-6hrfs\" (UID: \"ca79d6cb-be65-4e2f-af47-752ffac53ce1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6hrfs" Nov 04 03:53:51.202429 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:51.202408 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca79d6cb-be65-4e2f-af47-752ffac53ce1-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-6hrfs\" (UID: \"ca79d6cb-be65-4e2f-af47-752ffac53ce1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6hrfs" Nov 04 03:53:51.215143 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:51.215113 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59qbp\" (UniqueName: \"kubernetes.io/projected/ca79d6cb-be65-4e2f-af47-752ffac53ce1-kube-api-access-59qbp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-6hrfs\" (UID: \"ca79d6cb-be65-4e2f-af47-752ffac53ce1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6hrfs" Nov 04 03:53:51.257943 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:51.257849 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-2d8lw" event={"ID":"b279d26f-2d92-4082-8edb-d1c2a26022aa","Type":"ContainerStarted","Data":"ba0a239d2cb5161fde87dae0c97cd28545d11b030217f94ab06e7c75245bb191"} Nov 04 03:53:51.258117 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:51.258000 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-2d8lw" Nov 04 03:53:51.278728 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:51.278670 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-7b7bb4bc77-2d8lw" podStartSLOduration=0.896920665 podStartE2EDuration="3.27865222s" totalImagesPullingTime="2.381731555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:53:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:53:51.277961765 +0000 UTC m=+229.316401729" watchObservedRunningTime="2026-11-04 03:53:51.27865222 +0000 UTC m=+229.317092184" Nov 04 03:53:51.322245 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:51.322207 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6hrfs" Nov 04 03:53:51.612691 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:51.612660 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6hrfs"] Nov 04 03:53:51.616551 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:53:51.616518 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca79d6cb_be65_4e2f_af47_752ffac53ce1.slice/crio-193d258f2096cb3efc28cde2a3368faf9ff9c7c0f7d17cf453c4e9f708ea18d1 WatchSource:0}: Error finding container 193d258f2096cb3efc28cde2a3368faf9ff9c7c0f7d17cf453c4e9f708ea18d1: Status 404 returned error can't find the container with id 193d258f2096cb3efc28cde2a3368faf9ff9c7c0f7d17cf453c4e9f708ea18d1 Nov 04 03:53:52.263196 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:52.263161 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6hrfs" event={"ID":"ca79d6cb-be65-4e2f-af47-752ffac53ce1","Type":"ContainerStarted","Data":"193d258f2096cb3efc28cde2a3368faf9ff9c7c0f7d17cf453c4e9f708ea18d1"} Nov 04 03:53:52.264826 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:52.264795 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-jrjlz" event={"ID":"ddc1e44a-faa9-4022-a73b-4e1f87ee1210","Type":"ContainerStarted","Data":"1fc6e8ed896b92966f1eae404a76fcd2a1872558e9f93eb6a848f02a7760edf5"} Nov 04 03:53:52.265207 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:52.265180 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-jrjlz" Nov 04 03:53:52.294409 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:52.294358 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-jrjlz" podStartSLOduration=1.519442736 podStartE2EDuration="10.29434276s" totalImagesPullingTime="8.774900024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:53:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:53:52.291952149 +0000 UTC m=+230.330392113" watchObservedRunningTime="2026-11-04 03:53:52.29434276 +0000 UTC m=+230.332782702" Nov 04 03:53:53.218651 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:53.218619 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-jtnnz" Nov 04 03:53:55.280977 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:55.280942 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6hrfs" event={"ID":"ca79d6cb-be65-4e2f-af47-752ffac53ce1","Type":"ContainerStarted","Data":"2791bca163cc4e1c0742ec080100e8c1196b0170e43728857452fb1b36c9d320"} Nov 04 03:53:55.323979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:53:55.323904 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6hrfs" podStartSLOduration=2.352948781 podStartE2EDuration="5.323888167s" totalImagesPullingTime="2.970939386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:53:55.323365144 +0000 UTC m=+233.361805135" watchObservedRunningTime="2026-11-04 03:53:55.323888167 +0000 UTC m=+233.362328130" Nov 04 03:54:02.266983 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:02.266949 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-2d8lw" Nov 04 03:54:03.272414 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:03.272374 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-jrjlz" Nov 04 03:54:03.646536 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:03.646500 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2jdbl"] Nov 04 03:54:03.649747 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:03.649728 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2jdbl" Nov 04 03:54:03.664818 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:03.664791 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2jdbl"] Nov 04 03:54:03.724621 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:03.724581 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5qkn\" (UniqueName: \"kubernetes.io/projected/75ff9606-9236-43ed-b04f-d91f35ed4ea7-kube-api-access-r5qkn\") pod \"kuadrant-operator-controller-manager-678d87766f-2jdbl\" (UID: \"75ff9606-9236-43ed-b04f-d91f35ed4ea7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2jdbl" Nov 04 03:54:03.724863 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:03.724647 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/75ff9606-9236-43ed-b04f-d91f35ed4ea7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-2jdbl\" (UID: \"75ff9606-9236-43ed-b04f-d91f35ed4ea7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2jdbl" Nov 04 03:54:03.825464 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:03.825404 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5qkn\" (UniqueName: \"kubernetes.io/projected/75ff9606-9236-43ed-b04f-d91f35ed4ea7-kube-api-access-r5qkn\") pod \"kuadrant-operator-controller-manager-678d87766f-2jdbl\" (UID: \"75ff9606-9236-43ed-b04f-d91f35ed4ea7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2jdbl" Nov 04 03:54:03.825692 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:03.825515 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/75ff9606-9236-43ed-b04f-d91f35ed4ea7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-2jdbl\" (UID: \"75ff9606-9236-43ed-b04f-d91f35ed4ea7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2jdbl" Nov 04 03:54:03.825976 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:03.825954 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/75ff9606-9236-43ed-b04f-d91f35ed4ea7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-2jdbl\" (UID: \"75ff9606-9236-43ed-b04f-d91f35ed4ea7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2jdbl" Nov 04 03:54:03.838359 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:03.838336 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5qkn\" (UniqueName: \"kubernetes.io/projected/75ff9606-9236-43ed-b04f-d91f35ed4ea7-kube-api-access-r5qkn\") pod \"kuadrant-operator-controller-manager-678d87766f-2jdbl\" (UID: \"75ff9606-9236-43ed-b04f-d91f35ed4ea7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2jdbl" Nov 04 03:54:03.961000 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:03.960907 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2jdbl" Nov 04 03:54:04.246052 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:04.245858 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2jdbl"] Nov 04 03:54:04.247958 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:54:04.247932 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75ff9606_9236_43ed_b04f_d91f35ed4ea7.slice/crio-fa176931ac52b980d1a3cf06e86da384e2647bb8696f695662eb6e53a631f209 WatchSource:0}: Error finding container fa176931ac52b980d1a3cf06e86da384e2647bb8696f695662eb6e53a631f209: Status 404 returned error can't find the container with id fa176931ac52b980d1a3cf06e86da384e2647bb8696f695662eb6e53a631f209 Nov 04 03:54:04.324127 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:04.324089 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2jdbl" event={"ID":"75ff9606-9236-43ed-b04f-d91f35ed4ea7","Type":"ContainerStarted","Data":"c8ea35ed5d89d095b509d4d6d073292c985fc82a20b26d0ac3b008f7d043ee08"} Nov 04 03:54:04.324524 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:04.324147 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2jdbl" event={"ID":"75ff9606-9236-43ed-b04f-d91f35ed4ea7","Type":"ContainerStarted","Data":"fa176931ac52b980d1a3cf06e86da384e2647bb8696f695662eb6e53a631f209"} Nov 04 03:54:04.324524 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:04.324204 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2jdbl" Nov 04 03:54:04.352554 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:04.352500 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2jdbl" podStartSLOduration=1.35248606 podStartE2EDuration="1.35248606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:54:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:54:04.351469262 +0000 UTC m=+242.389909228" watchObservedRunningTime="2026-11-04 03:54:04.35248606 +0000 UTC m=+242.390926023" Nov 04 03:54:12.104516 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.104474 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2"] Nov 04 03:54:12.106965 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.106944 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.112465 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.112434 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-shmdl\"" Nov 04 03:54:12.121734 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.121705 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2"] Nov 04 03:54:12.202104 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.202060 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8873cfa-cbb1-4b2b-aa61-fca152bee660-istio-token\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.202104 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.202106 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8873cfa-cbb1-4b2b-aa61-fca152bee660-istio-data\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.202314 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.202248 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8873cfa-cbb1-4b2b-aa61-fca152bee660-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.202357 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.202326 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8873cfa-cbb1-4b2b-aa61-fca152bee660-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.202357 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.202352 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8873cfa-cbb1-4b2b-aa61-fca152bee660-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.202431 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.202374 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8873cfa-cbb1-4b2b-aa61-fca152bee660-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.202539 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.202515 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnvs7\" (UniqueName: \"kubernetes.io/projected/e8873cfa-cbb1-4b2b-aa61-fca152bee660-kube-api-access-cnvs7\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.202619 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.202604 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8873cfa-cbb1-4b2b-aa61-fca152bee660-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.202658 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.202626 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8873cfa-cbb1-4b2b-aa61-fca152bee660-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.303428 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.303392 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8873cfa-cbb1-4b2b-aa61-fca152bee660-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.303428 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.303432 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8873cfa-cbb1-4b2b-aa61-fca152bee660-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.303689 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.303452 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8873cfa-cbb1-4b2b-aa61-fca152bee660-istio-token\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.303689 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.303624 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8873cfa-cbb1-4b2b-aa61-fca152bee660-istio-data\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.303820 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.303710 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8873cfa-cbb1-4b2b-aa61-fca152bee660-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.303820 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.303771 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8873cfa-cbb1-4b2b-aa61-fca152bee660-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.303820 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.303813 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8873cfa-cbb1-4b2b-aa61-fca152bee660-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.303979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.303843 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8873cfa-cbb1-4b2b-aa61-fca152bee660-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.303979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.303883 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnvs7\" (UniqueName: \"kubernetes.io/projected/e8873cfa-cbb1-4b2b-aa61-fca152bee660-kube-api-access-cnvs7\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.303979 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.303892 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8873cfa-cbb1-4b2b-aa61-fca152bee660-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.304403 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.304225 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8873cfa-cbb1-4b2b-aa61-fca152bee660-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.304403 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.304390 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8873cfa-cbb1-4b2b-aa61-fca152bee660-istio-data\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.304681 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.304661 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8873cfa-cbb1-4b2b-aa61-fca152bee660-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.304775 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.304693 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8873cfa-cbb1-4b2b-aa61-fca152bee660-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.306449 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.306422 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8873cfa-cbb1-4b2b-aa61-fca152bee660-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.306645 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.306621 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8873cfa-cbb1-4b2b-aa61-fca152bee660-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.324465 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.324420 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8873cfa-cbb1-4b2b-aa61-fca152bee660-istio-token\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.325387 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.325354 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnvs7\" (UniqueName: \"kubernetes.io/projected/e8873cfa-cbb1-4b2b-aa61-fca152bee660-kube-api-access-cnvs7\") pod \"data-science-gateway-data-science-gateway-class-597d745469j8tv2\" (UID: \"e8873cfa-cbb1-4b2b-aa61-fca152bee660\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.420160 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.420075 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:12.699231 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.699132 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2"] Nov 04 03:54:12.702083 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:54:12.702050 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8873cfa_cbb1_4b2b_aa61_fca152bee660.slice/crio-309c6508481ec2c3ad3e1404220f40975bc41e5dc32df5ada6d81bdb5ad39419 WatchSource:0}: Error finding container 309c6508481ec2c3ad3e1404220f40975bc41e5dc32df5ada6d81bdb5ad39419: Status 404 returned error can't find the container with id 309c6508481ec2c3ad3e1404220f40975bc41e5dc32df5ada6d81bdb5ad39419 Nov 04 03:54:12.704349 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.704315 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:54:12.704449 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.704385 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:54:12.704449 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:12.704418 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:54:13.132497 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:13.132461 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-zf6m4"] Nov 04 03:54:13.134684 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:13.134663 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-zf6m4" Nov 04 03:54:13.142343 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:13.142317 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Nov 04 03:54:13.156868 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:13.156840 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-zf6m4"] Nov 04 03:54:13.185820 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:13.185746 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-zf6m4"] Nov 04 03:54:13.212810 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:13.212782 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5d4vv\"" Nov 04 03:54:13.313641 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:13.313604 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sspw\" (UniqueName: \"kubernetes.io/projected/343c0b0b-406a-4c05-b4dd-5b908f8c8c32-kube-api-access-2sspw\") pod \"limitador-limitador-7c9b87d574-zf6m4\" (UID: \"343c0b0b-406a-4c05-b4dd-5b908f8c8c32\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-zf6m4" Nov 04 03:54:13.314012 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:13.313974 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/343c0b0b-406a-4c05-b4dd-5b908f8c8c32-config-file\") pod \"limitador-limitador-7c9b87d574-zf6m4\" (UID: \"343c0b0b-406a-4c05-b4dd-5b908f8c8c32\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-zf6m4" Nov 04 03:54:13.369181 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:13.369137 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" event={"ID":"e8873cfa-cbb1-4b2b-aa61-fca152bee660","Type":"ContainerStarted","Data":"9d2611d2f08b228eb0209b65d468749701ca656079f057f7ac86038ccbe1c5f3"} Nov 04 03:54:13.369350 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:13.369205 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" event={"ID":"e8873cfa-cbb1-4b2b-aa61-fca152bee660","Type":"ContainerStarted","Data":"309c6508481ec2c3ad3e1404220f40975bc41e5dc32df5ada6d81bdb5ad39419"} Nov 04 03:54:13.394457 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:13.394359 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" podStartSLOduration=1.394343907 podStartE2EDuration="1.394343907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:54:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:54:13.393079233 +0000 UTC m=+251.431519188" watchObservedRunningTime="2026-11-04 03:54:13.394343907 +0000 UTC m=+251.432783871" Nov 04 03:54:13.414842 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:13.414810 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sspw\" (UniqueName: \"kubernetes.io/projected/343c0b0b-406a-4c05-b4dd-5b908f8c8c32-kube-api-access-2sspw\") pod \"limitador-limitador-7c9b87d574-zf6m4\" (UID: \"343c0b0b-406a-4c05-b4dd-5b908f8c8c32\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-zf6m4" Nov 04 03:54:13.414983 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:13.414858 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/343c0b0b-406a-4c05-b4dd-5b908f8c8c32-config-file\") pod \"limitador-limitador-7c9b87d574-zf6m4\" (UID: \"343c0b0b-406a-4c05-b4dd-5b908f8c8c32\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-zf6m4" Nov 04 03:54:13.415455 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:13.415436 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/343c0b0b-406a-4c05-b4dd-5b908f8c8c32-config-file\") pod \"limitador-limitador-7c9b87d574-zf6m4\" (UID: \"343c0b0b-406a-4c05-b4dd-5b908f8c8c32\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-zf6m4" Nov 04 03:54:13.420787 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:13.420744 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:13.425393 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:13.425372 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:13.452978 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:13.452947 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sspw\" (UniqueName: \"kubernetes.io/projected/343c0b0b-406a-4c05-b4dd-5b908f8c8c32-kube-api-access-2sspw\") pod \"limitador-limitador-7c9b87d574-zf6m4\" (UID: \"343c0b0b-406a-4c05-b4dd-5b908f8c8c32\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-zf6m4" Nov 04 03:54:13.746195 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:13.746113 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-zf6m4" Nov 04 03:54:13.825225 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:13.825188 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-wq9mr"] Nov 04 03:54:13.827962 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:13.827935 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-wq9mr" Nov 04 03:54:13.837307 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:13.837277 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-wq9mr"] Nov 04 03:54:13.844931 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:13.844905 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jxzfk\"" Nov 04 03:54:13.919210 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:13.919167 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n27l5\" (UniqueName: \"kubernetes.io/projected/f84cbd2c-0897-43db-add0-b5d0657cde2e-kube-api-access-n27l5\") pod \"authorino-56c6c9ddd5-wq9mr\" (UID: \"f84cbd2c-0897-43db-add0-b5d0657cde2e\") " pod="kuadrant-system/authorino-56c6c9ddd5-wq9mr" Nov 04 03:54:14.020663 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:14.020568 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n27l5\" (UniqueName: \"kubernetes.io/projected/f84cbd2c-0897-43db-add0-b5d0657cde2e-kube-api-access-n27l5\") pod \"authorino-56c6c9ddd5-wq9mr\" (UID: \"f84cbd2c-0897-43db-add0-b5d0657cde2e\") " pod="kuadrant-system/authorino-56c6c9ddd5-wq9mr" Nov 04 03:54:14.022457 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:14.022412 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-zf6m4"] Nov 04 03:54:14.024621 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:54:14.024588 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod343c0b0b_406a_4c05_b4dd_5b908f8c8c32.slice/crio-64f7c04488e32696ef4fdda7f22ad16cb6524ae7993c0d50ece363ba12a65cb0 WatchSource:0}: Error finding container 64f7c04488e32696ef4fdda7f22ad16cb6524ae7993c0d50ece363ba12a65cb0: Status 404 returned error can't find the container with id 64f7c04488e32696ef4fdda7f22ad16cb6524ae7993c0d50ece363ba12a65cb0 Nov 04 03:54:14.032857 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:14.032833 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n27l5\" (UniqueName: \"kubernetes.io/projected/f84cbd2c-0897-43db-add0-b5d0657cde2e-kube-api-access-n27l5\") pod \"authorino-56c6c9ddd5-wq9mr\" (UID: \"f84cbd2c-0897-43db-add0-b5d0657cde2e\") " pod="kuadrant-system/authorino-56c6c9ddd5-wq9mr" Nov 04 03:54:14.160697 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:14.160663 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-wq9mr" Nov 04 03:54:14.374828 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:14.374788 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-zf6m4" event={"ID":"343c0b0b-406a-4c05-b4dd-5b908f8c8c32","Type":"ContainerStarted","Data":"64f7c04488e32696ef4fdda7f22ad16cb6524ae7993c0d50ece363ba12a65cb0"} Nov 04 03:54:14.375118 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:14.375099 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:14.376186 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:14.376166 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-597d745469j8tv2" Nov 04 03:54:14.439003 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:14.438979 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-wq9mr"] Nov 04 03:54:15.332537 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:15.332501 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2jdbl" Nov 04 03:54:15.384796 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:15.383828 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-wq9mr" event={"ID":"f84cbd2c-0897-43db-add0-b5d0657cde2e","Type":"ContainerStarted","Data":"7d6a9aae7860f56460d86d0b2794bf3955e9c10d16fd6db9597b66457d9f66d6"} Nov 04 03:54:15.395876 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:15.395842 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-jrjlz"] Nov 04 03:54:15.396106 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:15.396075 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-jrjlz" podUID="ddc1e44a-faa9-4022-a73b-4e1f87ee1210" containerName="manager" containerID="cri-o://1fc6e8ed896b92966f1eae404a76fcd2a1872558e9f93eb6a848f02a7760edf5" gracePeriod=10 Nov 04 03:54:15.639667 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:15.639638 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-jrjlz" Nov 04 03:54:15.737016 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:15.736984 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ddc1e44a-faa9-4022-a73b-4e1f87ee1210-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ddc1e44a-faa9-4022-a73b-4e1f87ee1210-extensions-socket-volume\") pod \"ddc1e44a-faa9-4022-a73b-4e1f87ee1210\" (UID: \"ddc1e44a-faa9-4022-a73b-4e1f87ee1210\") " Nov 04 03:54:15.737170 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:15.737099 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ddc1e44a-faa9-4022-a73b-4e1f87ee1210-kube-api-access-wmh9c\" (UniqueName: \"kubernetes.io/projected/ddc1e44a-faa9-4022-a73b-4e1f87ee1210-kube-api-access-wmh9c\") pod \"ddc1e44a-faa9-4022-a73b-4e1f87ee1210\" (UID: \"ddc1e44a-faa9-4022-a73b-4e1f87ee1210\") " Nov 04 03:54:15.737413 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:15.737389 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddc1e44a-faa9-4022-a73b-4e1f87ee1210-extensions-socket-volume" pod "ddc1e44a-faa9-4022-a73b-4e1f87ee1210" (UID: "ddc1e44a-faa9-4022-a73b-4e1f87ee1210"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 03:54:15.739421 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:15.739396 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddc1e44a-faa9-4022-a73b-4e1f87ee1210-kube-api-access-wmh9c" pod "ddc1e44a-faa9-4022-a73b-4e1f87ee1210" (UID: "ddc1e44a-faa9-4022-a73b-4e1f87ee1210"). InnerVolumeSpecName "kube-api-access-wmh9c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 03:54:15.838056 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:15.838019 6000 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ddc1e44a-faa9-4022-a73b-4e1f87ee1210-extensions-socket-volume\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:54:15.838056 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:15.838050 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmh9c\" (UniqueName: \"kubernetes.io/projected/ddc1e44a-faa9-4022-a73b-4e1f87ee1210-kube-api-access-wmh9c\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:54:16.395886 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:16.395745 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-zf6m4" event={"ID":"343c0b0b-406a-4c05-b4dd-5b908f8c8c32","Type":"ContainerStarted","Data":"512c1c5eedcc42e2e32cc95dc92c5270d8a604c84f006d5080e900af4efaa9c4"} Nov 04 03:54:16.395886 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:16.395883 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-zf6m4" Nov 04 03:54:16.397189 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:16.397162 6000 generic.go:356] "Generic (PLEG): container finished" podID="ddc1e44a-faa9-4022-a73b-4e1f87ee1210" containerID="1fc6e8ed896b92966f1eae404a76fcd2a1872558e9f93eb6a848f02a7760edf5" exitCode=0 Nov 04 03:54:16.397323 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:16.397210 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-jrjlz" Nov 04 03:54:16.397323 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:16.397233 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-jrjlz" event={"ID":"ddc1e44a-faa9-4022-a73b-4e1f87ee1210","Type":"ContainerDied","Data":"1fc6e8ed896b92966f1eae404a76fcd2a1872558e9f93eb6a848f02a7760edf5"} Nov 04 03:54:16.397323 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:16.397266 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-jrjlz" event={"ID":"ddc1e44a-faa9-4022-a73b-4e1f87ee1210","Type":"ContainerDied","Data":"eec3b0a2f08a2a42098328b9e87cb151c91d85aca8dad17668d53b521a41a1b1"} Nov 04 03:54:16.397323 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:16.397285 6000 scope.go:122] "RemoveContainer" containerID="1fc6e8ed896b92966f1eae404a76fcd2a1872558e9f93eb6a848f02a7760edf5" Nov 04 03:54:16.406152 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:16.406132 6000 scope.go:122] "RemoveContainer" containerID="1fc6e8ed896b92966f1eae404a76fcd2a1872558e9f93eb6a848f02a7760edf5" Nov 04 03:54:16.406439 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:54:16.406417 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fc6e8ed896b92966f1eae404a76fcd2a1872558e9f93eb6a848f02a7760edf5\": container with ID starting with 1fc6e8ed896b92966f1eae404a76fcd2a1872558e9f93eb6a848f02a7760edf5 not found: ID does not exist" containerID="1fc6e8ed896b92966f1eae404a76fcd2a1872558e9f93eb6a848f02a7760edf5" Nov 04 03:54:16.406540 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:16.406445 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fc6e8ed896b92966f1eae404a76fcd2a1872558e9f93eb6a848f02a7760edf5"} err="failed to get container status \"1fc6e8ed896b92966f1eae404a76fcd2a1872558e9f93eb6a848f02a7760edf5\": rpc error: code = NotFound desc = could not find container \"1fc6e8ed896b92966f1eae404a76fcd2a1872558e9f93eb6a848f02a7760edf5\": container with ID starting with 1fc6e8ed896b92966f1eae404a76fcd2a1872558e9f93eb6a848f02a7760edf5 not found: ID does not exist" Nov 04 03:54:16.416859 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:16.416813 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-zf6m4" podStartSLOduration=1.9528847520000001 podStartE2EDuration="3.416798547s" totalImagesPullingTime="1.463913795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:54:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:54:16.416129052 +0000 UTC m=+254.454569017" watchObservedRunningTime="2026-11-04 03:54:16.416798547 +0000 UTC m=+254.455238511" Nov 04 03:54:16.430981 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:16.430953 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-jrjlz"] Nov 04 03:54:16.434964 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:16.434936 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-jrjlz"] Nov 04 03:54:18.080529 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:18.080490 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ddc1e44a-faa9-4022-a73b-4e1f87ee1210" path="/var/lib/kubelet/pods/ddc1e44a-faa9-4022-a73b-4e1f87ee1210/volumes" Nov 04 03:54:20.747062 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:54:20.747023 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 03:54:27.404867 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:27.404831 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-zf6m4" Nov 04 03:54:28.458715 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:28.458676 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-wq9mr" event={"ID":"f84cbd2c-0897-43db-add0-b5d0657cde2e","Type":"ContainerStarted","Data":"7be2cf9b62a1daeecfbd75c0f9456a24eaf4e475264c956da061d9b2a3519aac"} Nov 04 03:54:28.476393 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:28.476328 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-56c6c9ddd5-wq9mr" podStartSLOduration=1.590363989 podStartE2EDuration="15.476310862s" totalImagesPullingTime="13.885946873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:54:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:54:28.47506102 +0000 UTC m=+266.513500987" watchObservedRunningTime="2026-11-04 03:54:28.476310862 +0000 UTC m=+266.514750825" Nov 04 03:54:40.039233 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:40.039196 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-2wcfs"] Nov 04 03:54:40.039639 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:40.039547 6000 cpu_manager.go:414] "containerMap: removing container" podUID="ddc1e44a-faa9-4022-a73b-4e1f87ee1210" containerName="manager" Nov 04 03:54:40.039639 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:40.039559 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ddc1e44a-faa9-4022-a73b-4e1f87ee1210" containerName="manager" Nov 04 03:54:40.039639 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:40.039628 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="ddc1e44a-faa9-4022-a73b-4e1f87ee1210" containerName="manager" Nov 04 03:54:40.041960 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:40.041937 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2wcfs" Nov 04 03:54:40.050298 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:40.050277 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Nov 04 03:54:40.050424 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:40.050324 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-c92xv\"" Nov 04 03:54:40.051511 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:40.051495 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 04 03:54:40.051578 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:40.051499 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 04 03:54:40.067302 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:40.067268 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-2wcfs"] Nov 04 03:54:40.149398 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:40.149315 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/dc3db230-5977-4b27-9938-3d5f62ac53ad-data\") pod \"postgres-7cdbf6dc44-2wcfs\" (UID: \"dc3db230-5977-4b27-9938-3d5f62ac53ad\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2wcfs" Nov 04 03:54:40.149569 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:40.149469 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p2bz\" (UniqueName: \"kubernetes.io/projected/dc3db230-5977-4b27-9938-3d5f62ac53ad-kube-api-access-5p2bz\") pod \"postgres-7cdbf6dc44-2wcfs\" (UID: \"dc3db230-5977-4b27-9938-3d5f62ac53ad\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2wcfs" Nov 04 03:54:40.250880 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:40.250829 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p2bz\" (UniqueName: \"kubernetes.io/projected/dc3db230-5977-4b27-9938-3d5f62ac53ad-kube-api-access-5p2bz\") pod \"postgres-7cdbf6dc44-2wcfs\" (UID: \"dc3db230-5977-4b27-9938-3d5f62ac53ad\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2wcfs" Nov 04 03:54:40.251060 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:40.250905 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/dc3db230-5977-4b27-9938-3d5f62ac53ad-data\") pod \"postgres-7cdbf6dc44-2wcfs\" (UID: \"dc3db230-5977-4b27-9938-3d5f62ac53ad\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2wcfs" Nov 04 03:54:40.251334 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:40.251309 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/dc3db230-5977-4b27-9938-3d5f62ac53ad-data\") pod \"postgres-7cdbf6dc44-2wcfs\" (UID: \"dc3db230-5977-4b27-9938-3d5f62ac53ad\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2wcfs" Nov 04 03:54:40.263630 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:40.263594 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p2bz\" (UniqueName: \"kubernetes.io/projected/dc3db230-5977-4b27-9938-3d5f62ac53ad-kube-api-access-5p2bz\") pod \"postgres-7cdbf6dc44-2wcfs\" (UID: \"dc3db230-5977-4b27-9938-3d5f62ac53ad\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2wcfs" Nov 04 03:54:40.357960 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:40.357916 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2wcfs" Nov 04 03:54:40.630737 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:40.630704 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-2wcfs"] Nov 04 03:54:40.633191 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:54:40.633147 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc3db230_5977_4b27_9938_3d5f62ac53ad.slice/crio-2e17694d99eae1e1a27f0c4fd1ba4a9d7773905ad3fc59ced09918cb92050140 WatchSource:0}: Error finding container 2e17694d99eae1e1a27f0c4fd1ba4a9d7773905ad3fc59ced09918cb92050140: Status 404 returned error can't find the container with id 2e17694d99eae1e1a27f0c4fd1ba4a9d7773905ad3fc59ced09918cb92050140 Nov 04 03:54:41.527805 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:41.527766 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2wcfs" event={"ID":"dc3db230-5977-4b27-9938-3d5f62ac53ad","Type":"ContainerStarted","Data":"2e17694d99eae1e1a27f0c4fd1ba4a9d7773905ad3fc59ced09918cb92050140"} Nov 04 03:54:47.571277 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:47.571239 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2wcfs" event={"ID":"dc3db230-5977-4b27-9938-3d5f62ac53ad","Type":"ContainerStarted","Data":"5efd24ff0070e02477af14a96f391d6313a90232ad148a737c5d1c9b38df7b28"} Nov 04 03:54:47.571697 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:47.571343 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2wcfs" Nov 04 03:54:47.591280 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:47.591227 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2wcfs" podStartSLOduration=1.572019668 podStartE2EDuration="7.591213516s" totalImagesPullingTime="6.019193848s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:54:47.590978256 +0000 UTC m=+285.629418220" watchObservedRunningTime="2026-11-04 03:54:47.591213516 +0000 UTC m=+285.629653474" Nov 04 03:54:50.747821 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:54:50.747778 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 03:54:53.606519 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:54:53.606487 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2wcfs" Nov 04 03:55:02.117442 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:02.117407 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zfgzh_42510c3d-0283-4901-9d23-a84793598ac9/ovn-acl-logging/0.log" Nov 04 03:55:02.117939 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:02.117469 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zfgzh_42510c3d-0283-4901-9d23-a84793598ac9/ovn-acl-logging/0.log" Nov 04 03:55:06.360822 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:06.360788 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-r66wn"] Nov 04 03:55:06.366976 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:06.366950 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-r66wn" Nov 04 03:55:06.376195 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:06.376160 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-r66wn"] Nov 04 03:55:06.380539 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:06.380508 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\"" Nov 04 03:55:06.380682 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:06.380544 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Nov 04 03:55:06.380682 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:06.380550 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\"" Nov 04 03:55:06.380682 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:06.380580 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Nov 04 03:55:06.390107 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:06.390078 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\"" Nov 04 03:55:06.392771 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:06.392732 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-lxgtd\"" Nov 04 03:55:06.394173 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:06.394142 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/0fa75f01-dc06-4030-a496-6a354acf20c2-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-r66wn\" (UID: \"0fa75f01-dc06-4030-a496-6a354acf20c2\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-r66wn" Nov 04 03:55:06.394312 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:06.394228 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0fa75f01-dc06-4030-a496-6a354acf20c2-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-r66wn\" (UID: \"0fa75f01-dc06-4030-a496-6a354acf20c2\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-r66wn" Nov 04 03:55:06.394374 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:06.394316 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0fa75f01-dc06-4030-a496-6a354acf20c2-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-r66wn\" (UID: \"0fa75f01-dc06-4030-a496-6a354acf20c2\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-r66wn" Nov 04 03:55:06.394428 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:06.394406 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9d9g\" (UniqueName: \"kubernetes.io/projected/0fa75f01-dc06-4030-a496-6a354acf20c2-kube-api-access-n9d9g\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-r66wn\" (UID: \"0fa75f01-dc06-4030-a496-6a354acf20c2\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-r66wn" Nov 04 03:55:06.495513 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:06.495469 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9d9g\" (UniqueName: \"kubernetes.io/projected/0fa75f01-dc06-4030-a496-6a354acf20c2-kube-api-access-n9d9g\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-r66wn\" (UID: \"0fa75f01-dc06-4030-a496-6a354acf20c2\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-r66wn" Nov 04 03:55:06.495690 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:06.495539 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/0fa75f01-dc06-4030-a496-6a354acf20c2-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-r66wn\" (UID: \"0fa75f01-dc06-4030-a496-6a354acf20c2\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-r66wn" Nov 04 03:55:06.495690 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:06.495593 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0fa75f01-dc06-4030-a496-6a354acf20c2-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-r66wn\" (UID: \"0fa75f01-dc06-4030-a496-6a354acf20c2\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-r66wn" Nov 04 03:55:06.495690 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:06.495667 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0fa75f01-dc06-4030-a496-6a354acf20c2-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-r66wn\" (UID: \"0fa75f01-dc06-4030-a496-6a354acf20c2\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-r66wn" Nov 04 03:55:06.495838 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:55:06.495821 6000 secret.go:189] Couldn't get secret opendatahub/usage-logs-tenancy-proxy-tls: secret "usage-logs-tenancy-proxy-tls" not found Nov 04 03:55:06.495917 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:55:06.495908 6000 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fa75f01-dc06-4030-a496-6a354acf20c2-tls-cert podName:0fa75f01-dc06-4030-a496-6a354acf20c2 nodeName:}" failed. No retries permitted until 2026-11-04 03:55:06.995891695 +0000 UTC m=+305.034331653 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-cert" (UniqueName: "kubernetes.io/secret/0fa75f01-dc06-4030-a496-6a354acf20c2-tls-cert") pod "usage-logs-tenancy-proxy-9c5d49bcd-r66wn" (UID: "0fa75f01-dc06-4030-a496-6a354acf20c2") : secret "usage-logs-tenancy-proxy-tls" not found Nov 04 03:55:06.496315 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:06.496297 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0fa75f01-dc06-4030-a496-6a354acf20c2-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-r66wn\" (UID: \"0fa75f01-dc06-4030-a496-6a354acf20c2\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-r66wn" Nov 04 03:55:06.496515 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:06.496495 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/0fa75f01-dc06-4030-a496-6a354acf20c2-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-r66wn\" (UID: \"0fa75f01-dc06-4030-a496-6a354acf20c2\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-r66wn" Nov 04 03:55:06.519582 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:06.519546 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9d9g\" (UniqueName: \"kubernetes.io/projected/0fa75f01-dc06-4030-a496-6a354acf20c2-kube-api-access-n9d9g\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-r66wn\" (UID: \"0fa75f01-dc06-4030-a496-6a354acf20c2\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-r66wn" Nov 04 03:55:07.000887 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:07.000846 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0fa75f01-dc06-4030-a496-6a354acf20c2-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-r66wn\" (UID: \"0fa75f01-dc06-4030-a496-6a354acf20c2\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-r66wn" Nov 04 03:55:07.003294 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:07.003263 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0fa75f01-dc06-4030-a496-6a354acf20c2-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-r66wn\" (UID: \"0fa75f01-dc06-4030-a496-6a354acf20c2\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-r66wn" Nov 04 03:55:07.282221 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:07.282130 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-r66wn" Nov 04 03:55:07.591528 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:07.591502 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-r66wn"] Nov 04 03:55:07.594240 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:55:07.594203 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fa75f01_dc06_4030_a496_6a354acf20c2.slice/crio-cc65204e2b642d755f540fed2536d6728725ed99defb627017f3cc3928493152 WatchSource:0}: Error finding container cc65204e2b642d755f540fed2536d6728725ed99defb627017f3cc3928493152: Status 404 returned error can't find the container with id cc65204e2b642d755f540fed2536d6728725ed99defb627017f3cc3928493152 Nov 04 03:55:07.669994 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:07.669962 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-r66wn" event={"ID":"0fa75f01-dc06-4030-a496-6a354acf20c2","Type":"ContainerStarted","Data":"cc65204e2b642d755f540fed2536d6728725ed99defb627017f3cc3928493152"} Nov 04 03:55:10.889010 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:10.888974 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-xjm8c"] Nov 04 03:55:10.893244 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:10.893218 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xjm8c" Nov 04 03:55:10.906165 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:10.906135 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-xjm8c"] Nov 04 03:55:10.919049 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:10.918997 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Nov 04 03:55:10.919559 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:10.919529 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-glr6c\"" Nov 04 03:55:10.935421 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:10.935321 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxv22\" (UniqueName: \"kubernetes.io/projected/a7c91115-7d73-49d2-8c8b-2f244b2776b3-kube-api-access-hxv22\") pod \"llmisvc-controller-manager-869d5dcc5d-xjm8c\" (UID: \"a7c91115-7d73-49d2-8c8b-2f244b2776b3\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xjm8c" Nov 04 03:55:10.936077 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:10.935955 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7c91115-7d73-49d2-8c8b-2f244b2776b3-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-xjm8c\" (UID: \"a7c91115-7d73-49d2-8c8b-2f244b2776b3\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xjm8c" Nov 04 03:55:11.037481 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:11.037435 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxv22\" (UniqueName: \"kubernetes.io/projected/a7c91115-7d73-49d2-8c8b-2f244b2776b3-kube-api-access-hxv22\") pod \"llmisvc-controller-manager-869d5dcc5d-xjm8c\" (UID: \"a7c91115-7d73-49d2-8c8b-2f244b2776b3\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xjm8c" Nov 04 03:55:11.037656 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:11.037542 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7c91115-7d73-49d2-8c8b-2f244b2776b3-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-xjm8c\" (UID: \"a7c91115-7d73-49d2-8c8b-2f244b2776b3\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xjm8c" Nov 04 03:55:11.041927 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:11.041895 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7c91115-7d73-49d2-8c8b-2f244b2776b3-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-xjm8c\" (UID: \"a7c91115-7d73-49d2-8c8b-2f244b2776b3\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xjm8c" Nov 04 03:55:11.063811 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:11.063725 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxv22\" (UniqueName: \"kubernetes.io/projected/a7c91115-7d73-49d2-8c8b-2f244b2776b3-kube-api-access-hxv22\") pod \"llmisvc-controller-manager-869d5dcc5d-xjm8c\" (UID: \"a7c91115-7d73-49d2-8c8b-2f244b2776b3\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xjm8c" Nov 04 03:55:11.212021 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:11.211941 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xjm8c" Nov 04 03:55:11.570528 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:11.570498 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-xjm8c"] Nov 04 03:55:11.572991 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:55:11.572955 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda7c91115_7d73_49d2_8c8b_2f244b2776b3.slice/crio-7329a4e8b3e8b3cc8b59ba23a46a110a83e3388ef813ebe8da256c912b0e8bb6 WatchSource:0}: Error finding container 7329a4e8b3e8b3cc8b59ba23a46a110a83e3388ef813ebe8da256c912b0e8bb6: Status 404 returned error can't find the container with id 7329a4e8b3e8b3cc8b59ba23a46a110a83e3388ef813ebe8da256c912b0e8bb6 Nov 04 03:55:11.696786 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:11.696663 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xjm8c" event={"ID":"a7c91115-7d73-49d2-8c8b-2f244b2776b3","Type":"ContainerStarted","Data":"7329a4e8b3e8b3cc8b59ba23a46a110a83e3388ef813ebe8da256c912b0e8bb6"} Nov 04 03:55:15.730228 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:15.730190 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xjm8c" event={"ID":"a7c91115-7d73-49d2-8c8b-2f244b2776b3","Type":"ContainerStarted","Data":"7840d2f6ebea4e42e8eb49e3d264634564515a23622d2aeba982a11769b654e6"} Nov 04 03:55:15.730728 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:15.730372 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xjm8c" Nov 04 03:55:15.795738 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:15.795686 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xjm8c" podStartSLOduration=2.1610286260000002 podStartE2EDuration="5.795671203s" totalImagesPullingTime="3.634642577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:55:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:55:15.793943866 +0000 UTC m=+313.832383831" watchObservedRunningTime="2026-11-04 03:55:15.795671203 +0000 UTC m=+313.834111166" Nov 04 03:55:20.751133 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:55:20.751062 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 03:55:21.767060 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:21.766961 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-r66wn" event={"ID":"0fa75f01-dc06-4030-a496-6a354acf20c2","Type":"ContainerStarted","Data":"8bca206b346fd755246dd7cf3869d1032d1e5e550dacbad7c99d7ca880ead780"} Nov 04 03:55:21.767464 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:21.767086 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-r66wn" Nov 04 03:55:21.788276 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:21.788230 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-r66wn" podStartSLOduration=1.984203065 podStartE2EDuration="15.788216167s" totalImagesPullingTime="13.804013102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:55:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:55:21.787646094 +0000 UTC m=+319.826086059" watchObservedRunningTime="2026-11-04 03:55:21.788216167 +0000 UTC m=+319.826656169" Nov 04 03:55:32.779251 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:32.779221 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-r66wn" Nov 04 03:55:37.824565 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:37.824525 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-6vj98"] Nov 04 03:55:37.828463 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:37.828444 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-674d648f7b-6vj98" Nov 04 03:55:37.835684 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:37.835600 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Nov 04 03:55:37.835684 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:37.835617 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Nov 04 03:55:37.835889 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:37.835794 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bwgrq\"" Nov 04 03:55:37.840585 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:37.840561 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-6vj98"] Nov 04 03:55:37.990599 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:37.990556 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b2rf\" (UniqueName: \"kubernetes.io/projected/1f0181de-0bc4-4736-a2ea-e412b5d9c828-kube-api-access-6b2rf\") pod \"maas-api-674d648f7b-6vj98\" (UID: \"1f0181de-0bc4-4736-a2ea-e412b5d9c828\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-6vj98" Nov 04 03:55:37.990599 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:37.990604 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f0181de-0bc4-4736-a2ea-e412b5d9c828-maas-api-tls\") pod \"maas-api-674d648f7b-6vj98\" (UID: \"1f0181de-0bc4-4736-a2ea-e412b5d9c828\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-6vj98" Nov 04 03:55:37.990856 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:37.990625 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f0181de-0bc4-4736-a2ea-e412b5d9c828-openshift-service-ca\") pod \"maas-api-674d648f7b-6vj98\" (UID: \"1f0181de-0bc4-4736-a2ea-e412b5d9c828\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-6vj98" Nov 04 03:55:37.990856 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:37.990795 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f0181de-0bc4-4736-a2ea-e412b5d9c828-metrics-certs\") pod \"maas-api-674d648f7b-6vj98\" (UID: \"1f0181de-0bc4-4736-a2ea-e412b5d9c828\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-6vj98" Nov 04 03:55:38.092236 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:38.092144 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b2rf\" (UniqueName: \"kubernetes.io/projected/1f0181de-0bc4-4736-a2ea-e412b5d9c828-kube-api-access-6b2rf\") pod \"maas-api-674d648f7b-6vj98\" (UID: \"1f0181de-0bc4-4736-a2ea-e412b5d9c828\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-6vj98" Nov 04 03:55:38.092236 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:38.092185 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f0181de-0bc4-4736-a2ea-e412b5d9c828-maas-api-tls\") pod \"maas-api-674d648f7b-6vj98\" (UID: \"1f0181de-0bc4-4736-a2ea-e412b5d9c828\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-6vj98" Nov 04 03:55:38.092236 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:38.092201 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f0181de-0bc4-4736-a2ea-e412b5d9c828-openshift-service-ca\") pod \"maas-api-674d648f7b-6vj98\" (UID: \"1f0181de-0bc4-4736-a2ea-e412b5d9c828\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-6vj98" Nov 04 03:55:38.092520 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:38.092250 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f0181de-0bc4-4736-a2ea-e412b5d9c828-metrics-certs\") pod \"maas-api-674d648f7b-6vj98\" (UID: \"1f0181de-0bc4-4736-a2ea-e412b5d9c828\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-6vj98" Nov 04 03:55:38.092967 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:38.092939 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f0181de-0bc4-4736-a2ea-e412b5d9c828-openshift-service-ca\") pod \"maas-api-674d648f7b-6vj98\" (UID: \"1f0181de-0bc4-4736-a2ea-e412b5d9c828\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-6vj98" Nov 04 03:55:38.094874 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:38.094848 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f0181de-0bc4-4736-a2ea-e412b5d9c828-metrics-certs\") pod \"maas-api-674d648f7b-6vj98\" (UID: \"1f0181de-0bc4-4736-a2ea-e412b5d9c828\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-6vj98" Nov 04 03:55:38.094997 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:38.094882 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f0181de-0bc4-4736-a2ea-e412b5d9c828-maas-api-tls\") pod \"maas-api-674d648f7b-6vj98\" (UID: \"1f0181de-0bc4-4736-a2ea-e412b5d9c828\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-6vj98" Nov 04 03:55:38.104628 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:38.104596 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b2rf\" (UniqueName: \"kubernetes.io/projected/1f0181de-0bc4-4736-a2ea-e412b5d9c828-kube-api-access-6b2rf\") pod \"maas-api-674d648f7b-6vj98\" (UID: \"1f0181de-0bc4-4736-a2ea-e412b5d9c828\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-6vj98" Nov 04 03:55:38.142642 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:38.142601 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-674d648f7b-6vj98" Nov 04 03:55:38.420531 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:38.420348 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-6vj98"] Nov 04 03:55:38.423225 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:55:38.423200 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f0181de_0bc4_4736_a2ea_e412b5d9c828.slice/crio-a98c972321df9253b165f0512f1d6fe1c70af0aabc31d2062894f8ddb5673399 WatchSource:0}: Error finding container a98c972321df9253b165f0512f1d6fe1c70af0aabc31d2062894f8ddb5673399: Status 404 returned error can't find the container with id a98c972321df9253b165f0512f1d6fe1c70af0aabc31d2062894f8ddb5673399 Nov 04 03:55:38.700719 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:38.700638 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-zf6m4"] Nov 04 03:55:38.863603 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:38.863559 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-674d648f7b-6vj98" event={"ID":"1f0181de-0bc4-4736-a2ea-e412b5d9c828","Type":"ContainerStarted","Data":"a98c972321df9253b165f0512f1d6fe1c70af0aabc31d2062894f8ddb5673399"} Nov 04 03:55:41.881550 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:41.881455 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-674d648f7b-6vj98" event={"ID":"1f0181de-0bc4-4736-a2ea-e412b5d9c828","Type":"ContainerStarted","Data":"6acdd309268246668919bb5e72294ec0dd9d8ba1ed49d70a4f87c802eecbab12"} Nov 04 03:55:41.881550 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:41.881536 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-674d648f7b-6vj98" Nov 04 03:55:41.903161 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:41.903103 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-674d648f7b-6vj98" podStartSLOduration=1.7321410259999999 podStartE2EDuration="4.903084274s" totalImagesPullingTime="3.170943248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:55:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:55:41.90124874 +0000 UTC m=+339.939688703" watchObservedRunningTime="2026-11-04 03:55:41.903084274 +0000 UTC m=+339.941524285" Nov 04 03:55:46.743210 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:46.742925 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xjm8c" Nov 04 03:55:47.893196 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:47.893158 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-674d648f7b-6vj98" Nov 04 03:55:48.727017 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:48.726978 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-kp9rh"] Nov 04 03:55:48.749086 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:48.748994 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-kp9rh"] Nov 04 03:55:48.749086 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:48.749022 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-kp9rh" Nov 04 03:55:48.770247 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:48.770219 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Nov 04 03:55:48.809134 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:48.809098 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22czv\" (UniqueName: \"kubernetes.io/projected/d8736223-a73a-401b-8992-7dcaa0902207-kube-api-access-22czv\") pod \"authorino-6889cbd8c4-kp9rh\" (UID: \"d8736223-a73a-401b-8992-7dcaa0902207\") " pod="kuadrant-system/authorino-6889cbd8c4-kp9rh" Nov 04 03:55:48.809338 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:48.809176 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d8736223-a73a-401b-8992-7dcaa0902207-tls-cert\") pod \"authorino-6889cbd8c4-kp9rh\" (UID: \"d8736223-a73a-401b-8992-7dcaa0902207\") " pod="kuadrant-system/authorino-6889cbd8c4-kp9rh" Nov 04 03:55:48.910566 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:48.910473 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d8736223-a73a-401b-8992-7dcaa0902207-tls-cert\") pod \"authorino-6889cbd8c4-kp9rh\" (UID: \"d8736223-a73a-401b-8992-7dcaa0902207\") " pod="kuadrant-system/authorino-6889cbd8c4-kp9rh" Nov 04 03:55:48.911077 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:48.910575 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22czv\" (UniqueName: \"kubernetes.io/projected/d8736223-a73a-401b-8992-7dcaa0902207-kube-api-access-22czv\") pod \"authorino-6889cbd8c4-kp9rh\" (UID: \"d8736223-a73a-401b-8992-7dcaa0902207\") " pod="kuadrant-system/authorino-6889cbd8c4-kp9rh" Nov 04 03:55:48.913277 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:48.913251 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d8736223-a73a-401b-8992-7dcaa0902207-tls-cert\") pod \"authorino-6889cbd8c4-kp9rh\" (UID: \"d8736223-a73a-401b-8992-7dcaa0902207\") " pod="kuadrant-system/authorino-6889cbd8c4-kp9rh" Nov 04 03:55:48.924639 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:48.924605 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22czv\" (UniqueName: \"kubernetes.io/projected/d8736223-a73a-401b-8992-7dcaa0902207-kube-api-access-22czv\") pod \"authorino-6889cbd8c4-kp9rh\" (UID: \"d8736223-a73a-401b-8992-7dcaa0902207\") " pod="kuadrant-system/authorino-6889cbd8c4-kp9rh" Nov 04 03:55:48.924997 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:48.924973 6000 generic.go:356] "Generic (PLEG): container finished" podID="fa196927-03ec-4015-8172-44098857d262" containerID="b92c0b4524a6f0c422cae13f885342196d0ec821f28607d2855a26f00162651e" exitCode=0 Nov 04 03:55:48.925060 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:48.925039 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" event={"ID":"fa196927-03ec-4015-8172-44098857d262","Type":"ContainerDied","Data":"b92c0b4524a6f0c422cae13f885342196d0ec821f28607d2855a26f00162651e"} Nov 04 03:55:48.925418 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:48.925403 6000 scope.go:122] "RemoveContainer" containerID="b92c0b4524a6f0c422cae13f885342196d0ec821f28607d2855a26f00162651e" Nov 04 03:55:48.926653 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:48.926615 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:55:48.926741 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:48.926698 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:55:48.926741 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:48.926736 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:55:49.033373 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:49.033341 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-kp9rh"] Nov 04 03:55:49.033583 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:49.033569 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-kp9rh" Nov 04 03:55:49.065954 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:49.065918 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-684fb9c484-8sf77"] Nov 04 03:55:49.076677 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:49.076631 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-684fb9c484-8sf77" Nov 04 03:55:49.083295 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:49.083224 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-684fb9c484-8sf77"] Nov 04 03:55:49.112279 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:49.112249 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a0e38f1a-8276-487e-ab82-d7ae4bf0b253-tls-cert\") pod \"authorino-684fb9c484-8sf77\" (UID: \"a0e38f1a-8276-487e-ab82-d7ae4bf0b253\") " pod="kuadrant-system/authorino-684fb9c484-8sf77" Nov 04 03:55:49.112509 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:49.112492 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xjsl\" (UniqueName: \"kubernetes.io/projected/a0e38f1a-8276-487e-ab82-d7ae4bf0b253-kube-api-access-6xjsl\") pod \"authorino-684fb9c484-8sf77\" (UID: \"a0e38f1a-8276-487e-ab82-d7ae4bf0b253\") " pod="kuadrant-system/authorino-684fb9c484-8sf77" Nov 04 03:55:49.213654 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:49.213552 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xjsl\" (UniqueName: \"kubernetes.io/projected/a0e38f1a-8276-487e-ab82-d7ae4bf0b253-kube-api-access-6xjsl\") pod \"authorino-684fb9c484-8sf77\" (UID: \"a0e38f1a-8276-487e-ab82-d7ae4bf0b253\") " pod="kuadrant-system/authorino-684fb9c484-8sf77" Nov 04 03:55:49.213654 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:49.213641 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a0e38f1a-8276-487e-ab82-d7ae4bf0b253-tls-cert\") pod \"authorino-684fb9c484-8sf77\" (UID: \"a0e38f1a-8276-487e-ab82-d7ae4bf0b253\") " pod="kuadrant-system/authorino-684fb9c484-8sf77" Nov 04 03:55:49.216317 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:49.216286 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a0e38f1a-8276-487e-ab82-d7ae4bf0b253-tls-cert\") pod \"authorino-684fb9c484-8sf77\" (UID: \"a0e38f1a-8276-487e-ab82-d7ae4bf0b253\") " pod="kuadrant-system/authorino-684fb9c484-8sf77" Nov 04 03:55:49.227112 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:49.227082 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xjsl\" (UniqueName: \"kubernetes.io/projected/a0e38f1a-8276-487e-ab82-d7ae4bf0b253-kube-api-access-6xjsl\") pod \"authorino-684fb9c484-8sf77\" (UID: \"a0e38f1a-8276-487e-ab82-d7ae4bf0b253\") " pod="kuadrant-system/authorino-684fb9c484-8sf77" Nov 04 03:55:49.331574 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:49.331521 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-kp9rh"] Nov 04 03:55:49.424085 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:49.423797 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-684fb9c484-8sf77" Nov 04 03:55:49.724780 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:49.716731 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-684fb9c484-8sf77"] Nov 04 03:55:49.738736 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:55:49.738656 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda0e38f1a_8276_487e_ab82_d7ae4bf0b253.slice/crio-d313ac8b8e733384e7f45134d23d6024bd5b4101de8ec9536eb938f37ba4ef49 WatchSource:0}: Error finding container d313ac8b8e733384e7f45134d23d6024bd5b4101de8ec9536eb938f37ba4ef49: Status 404 returned error can't find the container with id d313ac8b8e733384e7f45134d23d6024bd5b4101de8ec9536eb938f37ba4ef49 Nov 04 03:55:49.937511 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:49.937470 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" event={"ID":"fa196927-03ec-4015-8172-44098857d262","Type":"ContainerStarted","Data":"95179d74ef01e316fc6ba8df0c9b0110409db46033b1b1949c71cf79c35166dc"} Nov 04 03:55:49.938991 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:49.938957 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-684fb9c484-8sf77" event={"ID":"a0e38f1a-8276-487e-ab82-d7ae4bf0b253","Type":"ContainerStarted","Data":"d313ac8b8e733384e7f45134d23d6024bd5b4101de8ec9536eb938f37ba4ef49"} Nov 04 03:55:49.940171 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:49.940145 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6889cbd8c4-kp9rh" event={"ID":"d8736223-a73a-401b-8992-7dcaa0902207","Type":"ContainerStarted","Data":"a02653b9e158f880909302457dc1fd3ffad72c38b4c7f4137ab6060748d42ff3"} Nov 04 03:55:50.004924 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:50.004835 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:55:50.006011 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:50.005985 6000 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-skgsr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body= Nov 04 03:55:50.006072 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:50.006038 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" podUID="fa196927-03ec-4015-8172-44098857d262" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" Nov 04 03:55:50.788424 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:55:50.788383 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 03:55:50.948240 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:50.948203 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6889cbd8c4-kp9rh" event={"ID":"d8736223-a73a-401b-8992-7dcaa0902207","Type":"ContainerStarted","Data":"22f3f2570cf7e37e6975e820f28fee7cf71cbc3bdfc63471c72e33e4c6c90925"} Nov 04 03:55:50.948666 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:50.948338 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-6889cbd8c4-kp9rh" podUID="d8736223-a73a-401b-8992-7dcaa0902207" containerName="authorino" containerID="cri-o://22f3f2570cf7e37e6975e820f28fee7cf71cbc3bdfc63471c72e33e4c6c90925" gracePeriod=30 Nov 04 03:55:50.950533 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:50.950484 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-684fb9c484-8sf77" event={"ID":"a0e38f1a-8276-487e-ab82-d7ae4bf0b253","Type":"ContainerStarted","Data":"e8084575197e6523add0023e5f1e6c48b2ea9e97025ac6c0e4159b08b31d28fb"} Nov 04 03:55:50.971637 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:50.971566 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-6889cbd8c4-kp9rh" podStartSLOduration=1.8668259630000001 podStartE2EDuration="2.971552644s" totalImagesPullingTime="1.104726681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:55:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:55:50.969496228 +0000 UTC m=+349.007936192" watchObservedRunningTime="2026-11-04 03:55:50.971552644 +0000 UTC m=+349.009992646" Nov 04 03:55:50.989650 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:50.989550 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-684fb9c484-8sf77" podStartSLOduration=1.005422688 podStartE2EDuration="1.989533693s" totalImagesPullingTime="984.111005ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:55:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:55:50.988325566 +0000 UTC m=+349.026765531" watchObservedRunningTime="2026-11-04 03:55:50.989533693 +0000 UTC m=+349.027973657" Nov 04 03:55:51.006231 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.006196 6000 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-skgsr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body= Nov 04 03:55:51.006394 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.006263 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" podUID="fa196927-03ec-4015-8172-44098857d262" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" Nov 04 03:55:51.045794 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.036637 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-wq9mr"] Nov 04 03:55:51.046144 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.046065 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-56c6c9ddd5-wq9mr" podUID="f84cbd2c-0897-43db-add0-b5d0657cde2e" containerName="authorino" containerID="cri-o://7be2cf9b62a1daeecfbd75c0f9456a24eaf4e475264c956da061d9b2a3519aac" gracePeriod=30 Nov 04 03:55:51.294625 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.294491 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-kp9rh" Nov 04 03:55:51.412152 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.411836 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-wq9mr" Nov 04 03:55:51.454405 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.453997 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d8736223-a73a-401b-8992-7dcaa0902207-kube-api-access-22czv\" (UniqueName: \"kubernetes.io/projected/d8736223-a73a-401b-8992-7dcaa0902207-kube-api-access-22czv\") pod \"d8736223-a73a-401b-8992-7dcaa0902207\" (UID: \"d8736223-a73a-401b-8992-7dcaa0902207\") " Nov 04 03:55:51.454405 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.454136 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d8736223-a73a-401b-8992-7dcaa0902207-tls-cert\" (UniqueName: \"kubernetes.io/secret/d8736223-a73a-401b-8992-7dcaa0902207-tls-cert\") pod \"d8736223-a73a-401b-8992-7dcaa0902207\" (UID: \"d8736223-a73a-401b-8992-7dcaa0902207\") " Nov 04 03:55:51.456768 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.456713 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8736223-a73a-401b-8992-7dcaa0902207-kube-api-access-22czv" pod "d8736223-a73a-401b-8992-7dcaa0902207" (UID: "d8736223-a73a-401b-8992-7dcaa0902207"). InnerVolumeSpecName "kube-api-access-22czv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 03:55:51.472822 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.469018 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8736223-a73a-401b-8992-7dcaa0902207-tls-cert" pod "d8736223-a73a-401b-8992-7dcaa0902207" (UID: "d8736223-a73a-401b-8992-7dcaa0902207"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 03:55:51.555221 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.555187 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f84cbd2c-0897-43db-add0-b5d0657cde2e-kube-api-access-n27l5\" (UniqueName: \"kubernetes.io/projected/f84cbd2c-0897-43db-add0-b5d0657cde2e-kube-api-access-n27l5\") pod \"f84cbd2c-0897-43db-add0-b5d0657cde2e\" (UID: \"f84cbd2c-0897-43db-add0-b5d0657cde2e\") " Nov 04 03:55:51.555679 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.555664 6000 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d8736223-a73a-401b-8992-7dcaa0902207-tls-cert\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:55:51.555813 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.555796 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22czv\" (UniqueName: \"kubernetes.io/projected/d8736223-a73a-401b-8992-7dcaa0902207-kube-api-access-22czv\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:55:51.557357 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.557328 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f84cbd2c-0897-43db-add0-b5d0657cde2e-kube-api-access-n27l5" pod "f84cbd2c-0897-43db-add0-b5d0657cde2e" (UID: "f84cbd2c-0897-43db-add0-b5d0657cde2e"). InnerVolumeSpecName "kube-api-access-n27l5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 03:55:51.657044 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.656999 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n27l5\" (UniqueName: \"kubernetes.io/projected/f84cbd2c-0897-43db-add0-b5d0657cde2e-kube-api-access-n27l5\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:55:51.957284 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.957199 6000 generic.go:356] "Generic (PLEG): container finished" podID="d8736223-a73a-401b-8992-7dcaa0902207" containerID="22f3f2570cf7e37e6975e820f28fee7cf71cbc3bdfc63471c72e33e4c6c90925" exitCode=0 Nov 04 03:55:51.957284 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.957239 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-kp9rh" Nov 04 03:55:51.957851 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.957276 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6889cbd8c4-kp9rh" event={"ID":"d8736223-a73a-401b-8992-7dcaa0902207","Type":"ContainerDied","Data":"22f3f2570cf7e37e6975e820f28fee7cf71cbc3bdfc63471c72e33e4c6c90925"} Nov 04 03:55:51.957851 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.957318 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6889cbd8c4-kp9rh" event={"ID":"d8736223-a73a-401b-8992-7dcaa0902207","Type":"ContainerDied","Data":"a02653b9e158f880909302457dc1fd3ffad72c38b4c7f4137ab6060748d42ff3"} Nov 04 03:55:51.957851 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.957339 6000 scope.go:122] "RemoveContainer" containerID="22f3f2570cf7e37e6975e820f28fee7cf71cbc3bdfc63471c72e33e4c6c90925" Nov 04 03:55:51.958818 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.958794 6000 generic.go:356] "Generic (PLEG): container finished" podID="f84cbd2c-0897-43db-add0-b5d0657cde2e" containerID="7be2cf9b62a1daeecfbd75c0f9456a24eaf4e475264c956da061d9b2a3519aac" exitCode=0 Nov 04 03:55:51.958923 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.958849 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-wq9mr" Nov 04 03:55:51.958923 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.958897 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-wq9mr" event={"ID":"f84cbd2c-0897-43db-add0-b5d0657cde2e","Type":"ContainerDied","Data":"7be2cf9b62a1daeecfbd75c0f9456a24eaf4e475264c956da061d9b2a3519aac"} Nov 04 03:55:51.959017 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.958940 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-wq9mr" event={"ID":"f84cbd2c-0897-43db-add0-b5d0657cde2e","Type":"ContainerDied","Data":"7d6a9aae7860f56460d86d0b2794bf3955e9c10d16fd6db9597b66457d9f66d6"} Nov 04 03:55:51.968272 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.968254 6000 scope.go:122] "RemoveContainer" containerID="22f3f2570cf7e37e6975e820f28fee7cf71cbc3bdfc63471c72e33e4c6c90925" Nov 04 03:55:51.968641 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:55:51.968613 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22f3f2570cf7e37e6975e820f28fee7cf71cbc3bdfc63471c72e33e4c6c90925\": container with ID starting with 22f3f2570cf7e37e6975e820f28fee7cf71cbc3bdfc63471c72e33e4c6c90925 not found: ID does not exist" containerID="22f3f2570cf7e37e6975e820f28fee7cf71cbc3bdfc63471c72e33e4c6c90925" Nov 04 03:55:51.968729 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.968653 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22f3f2570cf7e37e6975e820f28fee7cf71cbc3bdfc63471c72e33e4c6c90925"} err="failed to get container status \"22f3f2570cf7e37e6975e820f28fee7cf71cbc3bdfc63471c72e33e4c6c90925\": rpc error: code = NotFound desc = could not find container \"22f3f2570cf7e37e6975e820f28fee7cf71cbc3bdfc63471c72e33e4c6c90925\": container with ID starting with 22f3f2570cf7e37e6975e820f28fee7cf71cbc3bdfc63471c72e33e4c6c90925 not found: ID does not exist" Nov 04 03:55:51.968729 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.968677 6000 scope.go:122] "RemoveContainer" containerID="7be2cf9b62a1daeecfbd75c0f9456a24eaf4e475264c956da061d9b2a3519aac" Nov 04 03:55:51.977309 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.977285 6000 scope.go:122] "RemoveContainer" containerID="7be2cf9b62a1daeecfbd75c0f9456a24eaf4e475264c956da061d9b2a3519aac" Nov 04 03:55:51.977635 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:55:51.977613 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7be2cf9b62a1daeecfbd75c0f9456a24eaf4e475264c956da061d9b2a3519aac\": container with ID starting with 7be2cf9b62a1daeecfbd75c0f9456a24eaf4e475264c956da061d9b2a3519aac not found: ID does not exist" containerID="7be2cf9b62a1daeecfbd75c0f9456a24eaf4e475264c956da061d9b2a3519aac" Nov 04 03:55:51.977713 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.977646 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7be2cf9b62a1daeecfbd75c0f9456a24eaf4e475264c956da061d9b2a3519aac"} err="failed to get container status \"7be2cf9b62a1daeecfbd75c0f9456a24eaf4e475264c956da061d9b2a3519aac\": rpc error: code = NotFound desc = could not find container \"7be2cf9b62a1daeecfbd75c0f9456a24eaf4e475264c956da061d9b2a3519aac\": container with ID starting with 7be2cf9b62a1daeecfbd75c0f9456a24eaf4e475264c956da061d9b2a3519aac not found: ID does not exist" Nov 04 03:55:51.982018 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.981981 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-kp9rh"] Nov 04 03:55:51.986357 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.986331 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-kp9rh"] Nov 04 03:55:51.998067 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:51.998038 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-wq9mr"] Nov 04 03:55:52.004089 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:52.004063 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-wq9mr"] Nov 04 03:55:52.004415 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:52.004387 6000 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-skgsr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body= Nov 04 03:55:52.004518 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:52.004445 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" podUID="fa196927-03ec-4015-8172-44098857d262" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" Nov 04 03:55:52.084294 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:52.084259 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d8736223-a73a-401b-8992-7dcaa0902207" path="/var/lib/kubelet/pods/d8736223-a73a-401b-8992-7dcaa0902207/volumes" Nov 04 03:55:52.084593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:52.084575 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f84cbd2c-0897-43db-add0-b5d0657cde2e" path="/var/lib/kubelet/pods/f84cbd2c-0897-43db-add0-b5d0657cde2e/volumes" Nov 04 03:55:52.920565 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:52.918367 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:55:53.005110 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:53.005074 6000 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-skgsr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body= Nov 04 03:55:53.005504 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:53.005130 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" podUID="fa196927-03ec-4015-8172-44098857d262" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" Nov 04 03:55:54.004656 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:54.004620 6000 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-skgsr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body= Nov 04 03:55:54.004877 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:54.004680 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" podUID="fa196927-03ec-4015-8172-44098857d262" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" Nov 04 03:55:55.004426 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:55.004386 6000 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-skgsr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body= Nov 04 03:55:55.004956 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:55.004450 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" podUID="fa196927-03ec-4015-8172-44098857d262" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" Nov 04 03:55:56.008366 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:56.008335 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:55:56.009424 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:56.009400 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-skgsr" Nov 04 03:55:57.452772 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:57.452711 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-6vj98"] Nov 04 03:55:57.453246 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:57.453014 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-674d648f7b-6vj98" podUID="1f0181de-0bc4-4736-a2ea-e412b5d9c828" containerName="maas-api" containerID="cri-o://6acdd309268246668919bb5e72294ec0dd9d8ba1ed49d70a4f87c802eecbab12" gracePeriod=30 Nov 04 03:55:57.697664 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:57.697636 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-674d648f7b-6vj98" Nov 04 03:55:57.811413 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:57.811379 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f0181de-0bc4-4736-a2ea-e412b5d9c828-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f0181de-0bc4-4736-a2ea-e412b5d9c828-openshift-service-ca\") pod \"1f0181de-0bc4-4736-a2ea-e412b5d9c828\" (UID: \"1f0181de-0bc4-4736-a2ea-e412b5d9c828\") " Nov 04 03:55:57.811586 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:57.811438 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1f0181de-0bc4-4736-a2ea-e412b5d9c828-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f0181de-0bc4-4736-a2ea-e412b5d9c828-maas-api-tls\") pod \"1f0181de-0bc4-4736-a2ea-e412b5d9c828\" (UID: \"1f0181de-0bc4-4736-a2ea-e412b5d9c828\") " Nov 04 03:55:57.811586 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:57.811497 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1f0181de-0bc4-4736-a2ea-e412b5d9c828-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f0181de-0bc4-4736-a2ea-e412b5d9c828-metrics-certs\") pod \"1f0181de-0bc4-4736-a2ea-e412b5d9c828\" (UID: \"1f0181de-0bc4-4736-a2ea-e412b5d9c828\") " Nov 04 03:55:57.811586 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:57.811547 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f0181de-0bc4-4736-a2ea-e412b5d9c828-kube-api-access-6b2rf\" (UniqueName: \"kubernetes.io/projected/1f0181de-0bc4-4736-a2ea-e412b5d9c828-kube-api-access-6b2rf\") pod \"1f0181de-0bc4-4736-a2ea-e412b5d9c828\" (UID: \"1f0181de-0bc4-4736-a2ea-e412b5d9c828\") " Nov 04 03:55:57.811896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:57.811866 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f0181de-0bc4-4736-a2ea-e412b5d9c828-openshift-service-ca" pod "1f0181de-0bc4-4736-a2ea-e412b5d9c828" (UID: "1f0181de-0bc4-4736-a2ea-e412b5d9c828"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 03:55:57.813751 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:57.813722 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f0181de-0bc4-4736-a2ea-e412b5d9c828-metrics-certs" pod "1f0181de-0bc4-4736-a2ea-e412b5d9c828" (UID: "1f0181de-0bc4-4736-a2ea-e412b5d9c828"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 03:55:57.813888 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:57.813853 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f0181de-0bc4-4736-a2ea-e412b5d9c828-maas-api-tls" pod "1f0181de-0bc4-4736-a2ea-e412b5d9c828" (UID: "1f0181de-0bc4-4736-a2ea-e412b5d9c828"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 03:55:57.813888 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:57.813869 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f0181de-0bc4-4736-a2ea-e412b5d9c828-kube-api-access-6b2rf" pod "1f0181de-0bc4-4736-a2ea-e412b5d9c828" (UID: "1f0181de-0bc4-4736-a2ea-e412b5d9c828"). InnerVolumeSpecName "kube-api-access-6b2rf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 03:55:57.912135 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:57.912089 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6b2rf\" (UniqueName: \"kubernetes.io/projected/1f0181de-0bc4-4736-a2ea-e412b5d9c828-kube-api-access-6b2rf\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:55:57.912135 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:57.912127 6000 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f0181de-0bc4-4736-a2ea-e412b5d9c828-openshift-service-ca\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:55:57.912135 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:57.912137 6000 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f0181de-0bc4-4736-a2ea-e412b5d9c828-maas-api-tls\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:55:57.912135 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:57.912146 6000 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f0181de-0bc4-4736-a2ea-e412b5d9c828-metrics-certs\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:55:57.991286 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:57.991241 6000 generic.go:356] "Generic (PLEG): container finished" podID="1f0181de-0bc4-4736-a2ea-e412b5d9c828" containerID="6acdd309268246668919bb5e72294ec0dd9d8ba1ed49d70a4f87c802eecbab12" exitCode=0 Nov 04 03:55:57.991492 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:57.991300 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-674d648f7b-6vj98" Nov 04 03:55:57.991492 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:57.991332 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-674d648f7b-6vj98" event={"ID":"1f0181de-0bc4-4736-a2ea-e412b5d9c828","Type":"ContainerDied","Data":"6acdd309268246668919bb5e72294ec0dd9d8ba1ed49d70a4f87c802eecbab12"} Nov 04 03:55:57.991492 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:57.991405 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-674d648f7b-6vj98" event={"ID":"1f0181de-0bc4-4736-a2ea-e412b5d9c828","Type":"ContainerDied","Data":"a98c972321df9253b165f0512f1d6fe1c70af0aabc31d2062894f8ddb5673399"} Nov 04 03:55:57.991492 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:57.991435 6000 scope.go:122] "RemoveContainer" containerID="6acdd309268246668919bb5e72294ec0dd9d8ba1ed49d70a4f87c802eecbab12" Nov 04 03:55:58.001740 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:58.001715 6000 scope.go:122] "RemoveContainer" containerID="6acdd309268246668919bb5e72294ec0dd9d8ba1ed49d70a4f87c802eecbab12" Nov 04 03:55:58.002089 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:55:58.002069 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6acdd309268246668919bb5e72294ec0dd9d8ba1ed49d70a4f87c802eecbab12\": container with ID starting with 6acdd309268246668919bb5e72294ec0dd9d8ba1ed49d70a4f87c802eecbab12 not found: ID does not exist" containerID="6acdd309268246668919bb5e72294ec0dd9d8ba1ed49d70a4f87c802eecbab12" Nov 04 03:55:58.002138 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:58.002098 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6acdd309268246668919bb5e72294ec0dd9d8ba1ed49d70a4f87c802eecbab12"} err="failed to get container status \"6acdd309268246668919bb5e72294ec0dd9d8ba1ed49d70a4f87c802eecbab12\": rpc error: code = NotFound desc = could not find container \"6acdd309268246668919bb5e72294ec0dd9d8ba1ed49d70a4f87c802eecbab12\": container with ID starting with 6acdd309268246668919bb5e72294ec0dd9d8ba1ed49d70a4f87c802eecbab12 not found: ID does not exist" Nov 04 03:55:58.014995 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:58.014961 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-6vj98"] Nov 04 03:55:58.020245 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:58.020219 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-6vj98"] Nov 04 03:55:58.080828 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:55:58.080719 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f0181de-0bc4-4736-a2ea-e412b5d9c828" path="/var/lib/kubelet/pods/1f0181de-0bc4-4736-a2ea-e412b5d9c828/volumes" Nov 04 03:56:12.104389 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:12.104307 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-5xmx7"] Nov 04 03:56:12.104835 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:12.104702 6000 cpu_manager.go:414] "containerMap: removing container" podUID="1f0181de-0bc4-4736-a2ea-e412b5d9c828" containerName="maas-api" Nov 04 03:56:12.104835 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:12.104714 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f0181de-0bc4-4736-a2ea-e412b5d9c828" containerName="maas-api" Nov 04 03:56:12.104835 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:12.104741 6000 cpu_manager.go:414] "containerMap: removing container" podUID="f84cbd2c-0897-43db-add0-b5d0657cde2e" containerName="authorino" Nov 04 03:56:12.104835 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:12.104750 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f84cbd2c-0897-43db-add0-b5d0657cde2e" containerName="authorino" Nov 04 03:56:12.104835 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:12.104776 6000 cpu_manager.go:414] "containerMap: removing container" podUID="d8736223-a73a-401b-8992-7dcaa0902207" containerName="authorino" Nov 04 03:56:12.104835 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:12.104781 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d8736223-a73a-401b-8992-7dcaa0902207" containerName="authorino" Nov 04 03:56:12.105054 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:12.104845 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="d8736223-a73a-401b-8992-7dcaa0902207" containerName="authorino" Nov 04 03:56:12.105054 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:12.104855 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f0181de-0bc4-4736-a2ea-e412b5d9c828" containerName="maas-api" Nov 04 03:56:12.105054 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:12.104864 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="f84cbd2c-0897-43db-add0-b5d0657cde2e" containerName="authorino" Nov 04 03:56:12.107638 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:12.107617 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-5xmx7" Nov 04 03:56:12.119936 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:12.119912 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-5xmx7"] Nov 04 03:56:12.243942 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:12.243897 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qftxw\" (UniqueName: \"kubernetes.io/projected/6da524eb-f360-4de3-979e-d577dfae526d-kube-api-access-qftxw\") pod \"llmisvc-controller-manager-6466c554d8-5xmx7\" (UID: \"6da524eb-f360-4de3-979e-d577dfae526d\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-5xmx7" Nov 04 03:56:12.244124 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:12.244019 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6da524eb-f360-4de3-979e-d577dfae526d-cert\") pod \"llmisvc-controller-manager-6466c554d8-5xmx7\" (UID: \"6da524eb-f360-4de3-979e-d577dfae526d\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-5xmx7" Nov 04 03:56:12.344535 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:12.344488 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6da524eb-f360-4de3-979e-d577dfae526d-cert\") pod \"llmisvc-controller-manager-6466c554d8-5xmx7\" (UID: \"6da524eb-f360-4de3-979e-d577dfae526d\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-5xmx7" Nov 04 03:56:12.344739 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:12.344597 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qftxw\" (UniqueName: \"kubernetes.io/projected/6da524eb-f360-4de3-979e-d577dfae526d-kube-api-access-qftxw\") pod \"llmisvc-controller-manager-6466c554d8-5xmx7\" (UID: \"6da524eb-f360-4de3-979e-d577dfae526d\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-5xmx7" Nov 04 03:56:12.347276 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:12.347245 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6da524eb-f360-4de3-979e-d577dfae526d-cert\") pod \"llmisvc-controller-manager-6466c554d8-5xmx7\" (UID: \"6da524eb-f360-4de3-979e-d577dfae526d\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-5xmx7" Nov 04 03:56:12.357827 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:12.357742 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qftxw\" (UniqueName: \"kubernetes.io/projected/6da524eb-f360-4de3-979e-d577dfae526d-kube-api-access-qftxw\") pod \"llmisvc-controller-manager-6466c554d8-5xmx7\" (UID: \"6da524eb-f360-4de3-979e-d577dfae526d\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-5xmx7" Nov 04 03:56:12.420863 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:12.420826 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-5xmx7" Nov 04 03:56:12.704878 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:12.704850 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-5xmx7"] Nov 04 03:56:12.707142 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:56:12.707105 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6da524eb_f360_4de3_979e_d577dfae526d.slice/crio-ac21350fecaa28cd0c7ec5a8a0e246af821342e6ce0f46393a017cb049ba9bb4 WatchSource:0}: Error finding container ac21350fecaa28cd0c7ec5a8a0e246af821342e6ce0f46393a017cb049ba9bb4: Status 404 returned error can't find the container with id ac21350fecaa28cd0c7ec5a8a0e246af821342e6ce0f46393a017cb049ba9bb4 Nov 04 03:56:13.076645 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:13.076603 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-5xmx7" event={"ID":"6da524eb-f360-4de3-979e-d577dfae526d","Type":"ContainerStarted","Data":"ac21350fecaa28cd0c7ec5a8a0e246af821342e6ce0f46393a017cb049ba9bb4"} Nov 04 03:56:14.097418 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:14.097374 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-5xmx7" event={"ID":"6da524eb-f360-4de3-979e-d577dfae526d","Type":"ContainerStarted","Data":"50506c9cd717e11a960d7014f1a47754b16f2c895621c37dd3aaf6968bdab5f2"} Nov 04 03:56:14.098853 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:14.098829 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-5xmx7" Nov 04 03:56:14.124205 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:14.124138 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-5xmx7" podStartSLOduration=1.709573115 podStartE2EDuration="2.124118777s" totalImagesPullingTime="414.545662ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:56:14.123245625 +0000 UTC m=+372.161685590" watchObservedRunningTime="2026-11-04 03:56:14.124118777 +0000 UTC m=+372.162558746" Nov 04 03:56:20.747365 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:56:20.747326 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 03:56:46.109364 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:46.109333 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-5xmx7" Nov 04 03:56:46.162644 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:46.162607 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-xjm8c"] Nov 04 03:56:46.162959 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:46.162900 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xjm8c" podUID="a7c91115-7d73-49d2-8c8b-2f244b2776b3" containerName="manager" containerID="cri-o://7840d2f6ebea4e42e8eb49e3d264634564515a23622d2aeba982a11769b654e6" gracePeriod=30 Nov 04 03:56:46.419484 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:46.419457 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xjm8c" Nov 04 03:56:46.445111 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:46.445073 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7c91115-7d73-49d2-8c8b-2f244b2776b3-kube-api-access-hxv22\" (UniqueName: \"kubernetes.io/projected/a7c91115-7d73-49d2-8c8b-2f244b2776b3-kube-api-access-hxv22\") pod \"a7c91115-7d73-49d2-8c8b-2f244b2776b3\" (UID: \"a7c91115-7d73-49d2-8c8b-2f244b2776b3\") " Nov 04 03:56:46.445287 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:46.445227 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a7c91115-7d73-49d2-8c8b-2f244b2776b3-cert\" (UniqueName: \"kubernetes.io/secret/a7c91115-7d73-49d2-8c8b-2f244b2776b3-cert\") pod \"a7c91115-7d73-49d2-8c8b-2f244b2776b3\" (UID: \"a7c91115-7d73-49d2-8c8b-2f244b2776b3\") " Nov 04 03:56:46.447423 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:46.447392 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7c91115-7d73-49d2-8c8b-2f244b2776b3-cert" pod "a7c91115-7d73-49d2-8c8b-2f244b2776b3" (UID: "a7c91115-7d73-49d2-8c8b-2f244b2776b3"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 03:56:46.447604 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:46.447576 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7c91115-7d73-49d2-8c8b-2f244b2776b3-kube-api-access-hxv22" pod "a7c91115-7d73-49d2-8c8b-2f244b2776b3" (UID: "a7c91115-7d73-49d2-8c8b-2f244b2776b3"). InnerVolumeSpecName "kube-api-access-hxv22". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 03:56:46.546260 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:46.546224 6000 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7c91115-7d73-49d2-8c8b-2f244b2776b3-cert\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:56:46.546260 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:46.546254 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxv22\" (UniqueName: \"kubernetes.io/projected/a7c91115-7d73-49d2-8c8b-2f244b2776b3-kube-api-access-hxv22\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:56:47.267789 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:47.267733 6000 generic.go:356] "Generic (PLEG): container finished" podID="a7c91115-7d73-49d2-8c8b-2f244b2776b3" containerID="7840d2f6ebea4e42e8eb49e3d264634564515a23622d2aeba982a11769b654e6" exitCode=0 Nov 04 03:56:47.268245 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:47.267801 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xjm8c" Nov 04 03:56:47.268245 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:47.267822 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xjm8c" event={"ID":"a7c91115-7d73-49d2-8c8b-2f244b2776b3","Type":"ContainerDied","Data":"7840d2f6ebea4e42e8eb49e3d264634564515a23622d2aeba982a11769b654e6"} Nov 04 03:56:47.268245 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:47.267874 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xjm8c" event={"ID":"a7c91115-7d73-49d2-8c8b-2f244b2776b3","Type":"ContainerDied","Data":"7329a4e8b3e8b3cc8b59ba23a46a110a83e3388ef813ebe8da256c912b0e8bb6"} Nov 04 03:56:47.268245 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:47.267896 6000 scope.go:122] "RemoveContainer" containerID="7840d2f6ebea4e42e8eb49e3d264634564515a23622d2aeba982a11769b654e6" Nov 04 03:56:47.277537 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:47.277508 6000 scope.go:122] "RemoveContainer" containerID="7840d2f6ebea4e42e8eb49e3d264634564515a23622d2aeba982a11769b654e6" Nov 04 03:56:47.277911 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:56:47.277876 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7840d2f6ebea4e42e8eb49e3d264634564515a23622d2aeba982a11769b654e6\": container with ID starting with 7840d2f6ebea4e42e8eb49e3d264634564515a23622d2aeba982a11769b654e6 not found: ID does not exist" containerID="7840d2f6ebea4e42e8eb49e3d264634564515a23622d2aeba982a11769b654e6" Nov 04 03:56:47.278025 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:47.277929 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7840d2f6ebea4e42e8eb49e3d264634564515a23622d2aeba982a11769b654e6"} err="failed to get container status \"7840d2f6ebea4e42e8eb49e3d264634564515a23622d2aeba982a11769b654e6\": rpc error: code = NotFound desc = could not find container \"7840d2f6ebea4e42e8eb49e3d264634564515a23622d2aeba982a11769b654e6\": container with ID starting with 7840d2f6ebea4e42e8eb49e3d264634564515a23622d2aeba982a11769b654e6 not found: ID does not exist" Nov 04 03:56:47.305364 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:47.305332 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-xjm8c"] Nov 04 03:56:47.309478 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:47.309448 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-xjm8c"] Nov 04 03:56:48.081408 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:56:48.081362 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a7c91115-7d73-49d2-8c8b-2f244b2776b3" path="/var/lib/kubelet/pods/a7c91115-7d73-49d2-8c8b-2f244b2776b3/volumes" Nov 04 03:56:50.748343 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:56:50.748302 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 03:57:07.490478 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:07.490439 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-zf6m4"] Nov 04 03:57:09.278399 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:09.278362 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-zf6m4"] Nov 04 03:57:10.579252 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:10.579210 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-zf6m4"] Nov 04 03:57:12.544284 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:12.544238 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-zf6m4"] Nov 04 03:57:15.987743 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:15.987700 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-zf6m4"] Nov 04 03:57:20.749099 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:57:20.749053 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 03:57:20.810862 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:20.810821 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-zf6m4"] Nov 04 03:57:35.052866 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:35.052818 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-684fb9c484-8sf77"] Nov 04 03:57:35.053388 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:35.053080 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-684fb9c484-8sf77" podUID="a0e38f1a-8276-487e-ab82-d7ae4bf0b253" containerName="authorino" containerID="cri-o://e8084575197e6523add0023e5f1e6c48b2ea9e97025ac6c0e4159b08b31d28fb" gracePeriod=30 Nov 04 03:57:35.312277 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:35.312201 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-684fb9c484-8sf77" Nov 04 03:57:35.390905 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:35.390871 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a0e38f1a-8276-487e-ab82-d7ae4bf0b253-kube-api-access-6xjsl\" (UniqueName: \"kubernetes.io/projected/a0e38f1a-8276-487e-ab82-d7ae4bf0b253-kube-api-access-6xjsl\") pod \"a0e38f1a-8276-487e-ab82-d7ae4bf0b253\" (UID: \"a0e38f1a-8276-487e-ab82-d7ae4bf0b253\") " Nov 04 03:57:35.391111 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:35.390926 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a0e38f1a-8276-487e-ab82-d7ae4bf0b253-tls-cert\" (UniqueName: \"kubernetes.io/secret/a0e38f1a-8276-487e-ab82-d7ae4bf0b253-tls-cert\") pod \"a0e38f1a-8276-487e-ab82-d7ae4bf0b253\" (UID: \"a0e38f1a-8276-487e-ab82-d7ae4bf0b253\") " Nov 04 03:57:35.393308 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:35.393275 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0e38f1a-8276-487e-ab82-d7ae4bf0b253-kube-api-access-6xjsl" pod "a0e38f1a-8276-487e-ab82-d7ae4bf0b253" (UID: "a0e38f1a-8276-487e-ab82-d7ae4bf0b253"). InnerVolumeSpecName "kube-api-access-6xjsl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 03:57:35.401383 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:35.401351 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0e38f1a-8276-487e-ab82-d7ae4bf0b253-tls-cert" pod "a0e38f1a-8276-487e-ab82-d7ae4bf0b253" (UID: "a0e38f1a-8276-487e-ab82-d7ae4bf0b253"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 03:57:35.491977 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:35.491939 6000 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a0e38f1a-8276-487e-ab82-d7ae4bf0b253-tls-cert\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:57:35.491977 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:35.491974 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xjsl\" (UniqueName: \"kubernetes.io/projected/a0e38f1a-8276-487e-ab82-d7ae4bf0b253-kube-api-access-6xjsl\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 03:57:35.534510 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:35.534467 6000 generic.go:356] "Generic (PLEG): container finished" podID="a0e38f1a-8276-487e-ab82-d7ae4bf0b253" containerID="e8084575197e6523add0023e5f1e6c48b2ea9e97025ac6c0e4159b08b31d28fb" exitCode=0 Nov 04 03:57:35.534707 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:35.534516 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-684fb9c484-8sf77" Nov 04 03:57:35.534707 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:35.534547 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-684fb9c484-8sf77" event={"ID":"a0e38f1a-8276-487e-ab82-d7ae4bf0b253","Type":"ContainerDied","Data":"e8084575197e6523add0023e5f1e6c48b2ea9e97025ac6c0e4159b08b31d28fb"} Nov 04 03:57:35.534707 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:35.534607 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-684fb9c484-8sf77" event={"ID":"a0e38f1a-8276-487e-ab82-d7ae4bf0b253","Type":"ContainerDied","Data":"d313ac8b8e733384e7f45134d23d6024bd5b4101de8ec9536eb938f37ba4ef49"} Nov 04 03:57:35.534707 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:35.534630 6000 scope.go:122] "RemoveContainer" containerID="e8084575197e6523add0023e5f1e6c48b2ea9e97025ac6c0e4159b08b31d28fb" Nov 04 03:57:35.544451 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:35.544431 6000 scope.go:122] "RemoveContainer" containerID="e8084575197e6523add0023e5f1e6c48b2ea9e97025ac6c0e4159b08b31d28fb" Nov 04 03:57:35.544871 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:57:35.544846 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8084575197e6523add0023e5f1e6c48b2ea9e97025ac6c0e4159b08b31d28fb\": container with ID starting with e8084575197e6523add0023e5f1e6c48b2ea9e97025ac6c0e4159b08b31d28fb not found: ID does not exist" containerID="e8084575197e6523add0023e5f1e6c48b2ea9e97025ac6c0e4159b08b31d28fb" Nov 04 03:57:35.545786 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:35.544885 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8084575197e6523add0023e5f1e6c48b2ea9e97025ac6c0e4159b08b31d28fb"} err="failed to get container status \"e8084575197e6523add0023e5f1e6c48b2ea9e97025ac6c0e4159b08b31d28fb\": rpc error: code = NotFound desc = could not find container \"e8084575197e6523add0023e5f1e6c48b2ea9e97025ac6c0e4159b08b31d28fb\": container with ID starting with e8084575197e6523add0023e5f1e6c48b2ea9e97025ac6c0e4159b08b31d28fb not found: ID does not exist" Nov 04 03:57:35.566775 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:35.566653 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-684fb9c484-8sf77"] Nov 04 03:57:35.572886 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:35.572849 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-684fb9c484-8sf77"] Nov 04 03:57:36.080660 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:36.080617 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a0e38f1a-8276-487e-ab82-d7ae4bf0b253" path="/var/lib/kubelet/pods/a0e38f1a-8276-487e-ab82-d7ae4bf0b253/volumes" Nov 04 03:57:42.112102 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.111903 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2"] Nov 04 03:57:42.114674 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.113396 6000 cpu_manager.go:414] "containerMap: removing container" podUID="a7c91115-7d73-49d2-8c8b-2f244b2776b3" containerName="manager" Nov 04 03:57:42.114674 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.113426 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7c91115-7d73-49d2-8c8b-2f244b2776b3" containerName="manager" Nov 04 03:57:42.114674 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.113481 6000 cpu_manager.go:414] "containerMap: removing container" podUID="a0e38f1a-8276-487e-ab82-d7ae4bf0b253" containerName="authorino" Nov 04 03:57:42.114674 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.113489 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a0e38f1a-8276-487e-ab82-d7ae4bf0b253" containerName="authorino" Nov 04 03:57:42.114674 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.113653 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="a0e38f1a-8276-487e-ab82-d7ae4bf0b253" containerName="authorino" Nov 04 03:57:42.114674 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.113677 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7c91115-7d73-49d2-8c8b-2f244b2776b3" containerName="manager" Nov 04 03:57:42.119197 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.119171 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.137872 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.137832 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2"] Nov 04 03:57:42.154739 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.154698 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee7b70f9-1025-4144-b882-fb75f1159c3e-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.155322 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.154819 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee7b70f9-1025-4144-b882-fb75f1159c3e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.155322 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.154883 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee7b70f9-1025-4144-b882-fb75f1159c3e-istio-token\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.155322 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.154917 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee7b70f9-1025-4144-b882-fb75f1159c3e-workload-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.155322 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.155002 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee7b70f9-1025-4144-b882-fb75f1159c3e-workload-certs\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.155322 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.155208 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee7b70f9-1025-4144-b882-fb75f1159c3e-credential-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.155322 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.155300 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee7b70f9-1025-4144-b882-fb75f1159c3e-istio-data\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.156007 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.155339 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49t6s\" (UniqueName: \"kubernetes.io/projected/ee7b70f9-1025-4144-b882-fb75f1159c3e-kube-api-access-49t6s\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.156007 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.155365 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee7b70f9-1025-4144-b882-fb75f1159c3e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.156643 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.156620 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf"] Nov 04 03:57:42.160679 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.160656 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.198980 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.198942 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf"] Nov 04 03:57:42.255967 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.255919 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbpqm\" (UniqueName: \"kubernetes.io/projected/01a09535-2de8-4f06-846b-556b9db5e38b-kube-api-access-cbpqm\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.255967 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.255975 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee7b70f9-1025-4144-b882-fb75f1159c3e-istio-token\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.256261 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.255998 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01a09535-2de8-4f06-846b-556b9db5e38b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.256261 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.256089 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee7b70f9-1025-4144-b882-fb75f1159c3e-workload-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.256261 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.256223 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee7b70f9-1025-4144-b882-fb75f1159c3e-workload-certs\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.256416 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.256302 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee7b70f9-1025-4144-b882-fb75f1159c3e-credential-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.256416 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.256368 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee7b70f9-1025-4144-b882-fb75f1159c3e-istio-data\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.256416 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.256403 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-workload-certs\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.256570 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.256431 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.256570 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.256468 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01a09535-2de8-4f06-846b-556b9db5e38b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.256671 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.256581 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49t6s\" (UniqueName: \"kubernetes.io/projected/ee7b70f9-1025-4144-b882-fb75f1159c3e-kube-api-access-49t6s\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.256671 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.256627 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee7b70f9-1025-4144-b882-fb75f1159c3e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.256802 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.256670 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee7b70f9-1025-4144-b882-fb75f1159c3e-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.256802 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.256709 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee7b70f9-1025-4144-b882-fb75f1159c3e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.256802 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.256715 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee7b70f9-1025-4144-b882-fb75f1159c3e-credential-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.256802 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.256738 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-workload-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.256802 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.256670 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee7b70f9-1025-4144-b882-fb75f1159c3e-workload-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.256802 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.256750 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee7b70f9-1025-4144-b882-fb75f1159c3e-workload-certs\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.257107 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.256839 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-credential-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.257107 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.256871 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-istio-data\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.257107 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.256898 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01a09535-2de8-4f06-846b-556b9db5e38b-istio-token\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.257107 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.257043 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee7b70f9-1025-4144-b882-fb75f1159c3e-istio-data\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.257314 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.257297 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee7b70f9-1025-4144-b882-fb75f1159c3e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.259088 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.259060 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee7b70f9-1025-4144-b882-fb75f1159c3e-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.259249 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.259233 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee7b70f9-1025-4144-b882-fb75f1159c3e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.274418 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.274389 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49t6s\" (UniqueName: \"kubernetes.io/projected/ee7b70f9-1025-4144-b882-fb75f1159c3e-kube-api-access-49t6s\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.274576 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.274557 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee7b70f9-1025-4144-b882-fb75f1159c3e-istio-token\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-kkgr2\" (UID: \"ee7b70f9-1025-4144-b882-fb75f1159c3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.358007 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.357967 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-workload-certs\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.358007 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.358007 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.358251 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.358027 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01a09535-2de8-4f06-846b-556b9db5e38b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.358251 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.358066 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-workload-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.358251 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.358084 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-credential-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.358251 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.358106 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-istio-data\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.358251 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.358132 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01a09535-2de8-4f06-846b-556b9db5e38b-istio-token\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.358251 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.358158 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbpqm\" (UniqueName: \"kubernetes.io/projected/01a09535-2de8-4f06-846b-556b9db5e38b-kube-api-access-cbpqm\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.358251 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.358192 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01a09535-2de8-4f06-846b-556b9db5e38b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.358611 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.358451 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-workload-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.358611 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.358551 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-istio-data\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.358611 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.358569 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-workload-certs\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.358796 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.358774 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-credential-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.358849 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.358832 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01a09535-2de8-4f06-846b-556b9db5e38b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.360702 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.360678 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01a09535-2de8-4f06-846b-556b9db5e38b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.360866 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.360819 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.372006 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.371941 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbpqm\" (UniqueName: \"kubernetes.io/projected/01a09535-2de8-4f06-846b-556b9db5e38b-kube-api-access-cbpqm\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.377293 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.377267 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01a09535-2de8-4f06-846b-556b9db5e38b-istio-token\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-gnswf\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.436152 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.436102 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:42.473812 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.473738 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:42.784899 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.784811 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2"] Nov 04 03:57:42.788361 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.788336 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf"] Nov 04 03:57:42.789193 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:57:42.789169 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee7b70f9_1025_4144_b882_fb75f1159c3e.slice/crio-ce6890aa739568f93d07524015885b3e8670bac99658e78cc55577b664a861a4 WatchSource:0}: Error finding container ce6890aa739568f93d07524015885b3e8670bac99658e78cc55577b664a861a4: Status 404 returned error can't find the container with id ce6890aa739568f93d07524015885b3e8670bac99658e78cc55577b664a861a4 Nov 04 03:57:42.789980 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:57:42.789958 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01a09535_2de8_4f06_846b_556b9db5e38b.slice/crio-4335ff87952ef9471b74e35d39057ac84a3949f912b8204b4c12bfe3c22af5df WatchSource:0}: Error finding container 4335ff87952ef9471b74e35d39057ac84a3949f912b8204b4c12bfe3c22af5df: Status 404 returned error can't find the container with id 4335ff87952ef9471b74e35d39057ac84a3949f912b8204b4c12bfe3c22af5df Nov 04 03:57:42.791535 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.791481 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:57:42.791591 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.791551 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:57:42.791642 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.791579 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:57:42.792177 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.792141 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:57:42.792261 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.792223 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:57:42.792306 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:42.792269 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:57:43.583863 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:43.583824 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" event={"ID":"ee7b70f9-1025-4144-b882-fb75f1159c3e","Type":"ContainerStarted","Data":"c7038cca558b22df908e7a9b2940879308d7706a077bf11e8a233257b06117f5"} Nov 04 03:57:43.584319 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:43.583892 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" event={"ID":"ee7b70f9-1025-4144-b882-fb75f1159c3e","Type":"ContainerStarted","Data":"ce6890aa739568f93d07524015885b3e8670bac99658e78cc55577b664a861a4"} Nov 04 03:57:43.585605 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:43.585572 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" event={"ID":"01a09535-2de8-4f06-846b-556b9db5e38b","Type":"ContainerStarted","Data":"ed1fd7d34aed2eb2ad514c40d4ff506bb1a412471859d53e3d8faca54d3e1cb5"} Nov 04 03:57:43.585777 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:43.585631 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" event={"ID":"01a09535-2de8-4f06-846b-556b9db5e38b","Type":"ContainerStarted","Data":"4335ff87952ef9471b74e35d39057ac84a3949f912b8204b4c12bfe3c22af5df"} Nov 04 03:57:43.731176 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:43.730232 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" podStartSLOduration=1.730210888 podStartE2EDuration="1.730210888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:57:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:57:43.647914085 +0000 UTC m=+461.686354051" watchObservedRunningTime="2026-11-04 03:57:43.730210888 +0000 UTC m=+461.768650852" Nov 04 03:57:43.731176 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:43.730410 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" podStartSLOduration=1.730404209 podStartE2EDuration="1.730404209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:57:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:57:43.728272797 +0000 UTC m=+461.766712762" watchObservedRunningTime="2026-11-04 03:57:43.730404209 +0000 UTC m=+461.768844175" Nov 04 03:57:44.441436 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:44.441399 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:44.441868 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:44.441843 6000 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 04 03:57:44.442031 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:44.442006 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" podUID="ee7b70f9-1025-4144-b882-fb75f1159c3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 04 03:57:44.474362 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:44.474326 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:44.475567 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:44.475532 6000 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-gnswf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 04 03:57:44.475647 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:44.475594 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" podUID="01a09535-2de8-4f06-846b-556b9db5e38b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 04 03:57:45.438780 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:45.438276 6000 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 04 03:57:45.438780 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:45.438349 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" podUID="ee7b70f9-1025-4144-b882-fb75f1159c3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 04 03:57:45.475335 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:45.475296 6000 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-gnswf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 04 03:57:45.475647 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:45.475608 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" podUID="01a09535-2de8-4f06-846b-556b9db5e38b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 04 03:57:46.453795 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:46.453488 6000 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 04 03:57:46.453795 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:46.453553 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" podUID="ee7b70f9-1025-4144-b882-fb75f1159c3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 04 03:57:46.474825 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:46.474783 6000 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-gnswf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 04 03:57:46.474992 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:46.474847 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" podUID="01a09535-2de8-4f06-846b-556b9db5e38b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 04 03:57:47.438271 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:47.438228 6000 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 04 03:57:47.439352 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:47.439316 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" podUID="ee7b70f9-1025-4144-b882-fb75f1159c3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 04 03:57:47.475150 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:47.475111 6000 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-gnswf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 04 03:57:47.475525 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:47.475175 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" podUID="01a09535-2de8-4f06-846b-556b9db5e38b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 04 03:57:48.444782 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:48.442281 6000 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 04 03:57:48.444782 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:48.442366 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" podUID="ee7b70f9-1025-4144-b882-fb75f1159c3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 04 03:57:48.474649 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:48.474599 6000 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-gnswf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 04 03:57:48.474857 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:48.474656 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" podUID="01a09535-2de8-4f06-846b-556b9db5e38b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 04 03:57:49.437592 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:49.437545 6000 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 04 03:57:49.438098 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:49.437613 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" podUID="ee7b70f9-1025-4144-b882-fb75f1159c3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 04 03:57:49.475042 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:49.474997 6000 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-gnswf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 04 03:57:49.475210 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:49.475068 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" podUID="01a09535-2de8-4f06-846b-556b9db5e38b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 04 03:57:50.437187 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:50.437143 6000 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 04 03:57:50.437375 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:50.437210 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" podUID="ee7b70f9-1025-4144-b882-fb75f1159c3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 04 03:57:50.474628 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:50.474591 6000 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-gnswf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 04 03:57:50.475109 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:50.474651 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" podUID="01a09535-2de8-4f06-846b-556b9db5e38b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 04 03:57:50.763101 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:57:50.762997 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 03:57:52.436833 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:52.436775 6000 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 03:57:52.437409 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:52.436888 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" podUID="ee7b70f9-1025-4144-b882-fb75f1159c3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 03:57:52.475727 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:52.475683 6000 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-gnswf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 03:57:52.475929 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:52.475788 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" podUID="01a09535-2de8-4f06-846b-556b9db5e38b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 03:57:53.477945 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:53.477892 6000 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 03:57:53.478374 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:53.477951 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" podUID="ee7b70f9-1025-4144-b882-fb75f1159c3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 03:57:53.518419 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:53.517966 6000 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-gnswf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 03:57:53.518419 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:53.518045 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" podUID="01a09535-2de8-4f06-846b-556b9db5e38b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 03:57:54.097856 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:54.097808 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:54.098255 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:54.098228 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:54.099171 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:54.099123 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-kkgr2" Nov 04 03:57:54.519225 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:54.519120 6000 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-gnswf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 03:57:54.519697 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:54.519214 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" podUID="01a09535-2de8-4f06-846b-556b9db5e38b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 03:57:54.522826 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:54.522794 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:54.523240 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:54.523220 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:57:54.524089 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:57:54.524070 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 03:58:20.748101 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:58:20.748062 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 03:58:35.497307 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.497272 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m"] Nov 04 03:58:35.501035 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.501013 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.517147 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.517125 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-56a44f-openshift-default-dockercfg-7cnql\"" Nov 04 03:58:35.544366 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.544331 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m"] Nov 04 03:58:35.595861 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.595820 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-istio-data\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.596072 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.595910 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-istio-envoy\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.596072 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.595941 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f15a8b2c-4f91-4792-869d-26e339189978-istio-token\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.596072 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.595997 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-credential-socket\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.596072 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.596024 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f15a8b2c-4f91-4792-869d-26e339189978-istio-podinfo\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.596313 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.596074 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrl9t\" (UniqueName: \"kubernetes.io/projected/f15a8b2c-4f91-4792-869d-26e339189978-kube-api-access-wrl9t\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.596313 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.596131 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-workload-certs\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.596313 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.596167 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-workload-socket\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.596313 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.596196 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f15a8b2c-4f91-4792-869d-26e339189978-istiod-ca-cert\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.697168 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.697129 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-credential-socket\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.697168 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.697172 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f15a8b2c-4f91-4792-869d-26e339189978-istio-podinfo\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.697449 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.697218 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrl9t\" (UniqueName: \"kubernetes.io/projected/f15a8b2c-4f91-4792-869d-26e339189978-kube-api-access-wrl9t\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.697449 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.697286 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-workload-certs\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.697449 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.697321 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-workload-socket\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.697449 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.697347 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f15a8b2c-4f91-4792-869d-26e339189978-istiod-ca-cert\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.697449 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.697387 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-istio-data\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.697709 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.697456 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-istio-envoy\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.697709 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.697482 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f15a8b2c-4f91-4792-869d-26e339189978-istio-token\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.697709 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.697618 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-credential-socket\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.697709 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.697693 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-workload-certs\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.697958 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.697934 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-istio-data\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.698075 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.698056 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-workload-socket\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.698323 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.698294 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f15a8b2c-4f91-4792-869d-26e339189978-istiod-ca-cert\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.700071 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.700041 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f15a8b2c-4f91-4792-869d-26e339189978-istio-podinfo\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.700221 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.700198 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-istio-envoy\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.738642 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.738609 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f15a8b2c-4f91-4792-869d-26e339189978-istio-token\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.762173 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.762087 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrl9t\" (UniqueName: \"kubernetes.io/projected/f15a8b2c-4f91-4792-869d-26e339189978-kube-api-access-wrl9t\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-f866m\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.813961 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.813922 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:35.898809 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.898723 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w"] Nov 04 03:58:35.912444 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.912411 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:35.919544 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.919438 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w"] Nov 04 03:58:35.921372 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:35.921285 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-6a9cdb-openshift-default-dockercfg-t8tng\"" Nov 04 03:58:36.000703 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.000665 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-credential-socket\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.000926 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.000775 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-workload-certs\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.000926 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.000868 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5858d840-2308-4cf2-b7f3-9f73509bb196-istiod-ca-cert\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.000926 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.000912 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-podinfo\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.001098 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.000937 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-envoy\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.001098 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.001007 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlc4r\" (UniqueName: \"kubernetes.io/projected/5858d840-2308-4cf2-b7f3-9f73509bb196-kube-api-access-nlc4r\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.001098 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.001087 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-token\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.001245 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.001143 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-data\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.001245 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.001178 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-workload-socket\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.102287 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.102251 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-data\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.102287 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.102290 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-workload-socket\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.102577 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.102342 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-credential-socket\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.102577 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.102357 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-workload-certs\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.102577 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.102383 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5858d840-2308-4cf2-b7f3-9f73509bb196-istiod-ca-cert\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.102577 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.102406 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-podinfo\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.102577 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.102489 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-envoy\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.102577 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.102561 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlc4r\" (UniqueName: \"kubernetes.io/projected/5858d840-2308-4cf2-b7f3-9f73509bb196-kube-api-access-nlc4r\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.102906 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.102602 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-token\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.102906 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.102771 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-data\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.102906 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.102879 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-credential-socket\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.103221 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.103196 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-workload-certs\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.103356 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.103289 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-workload-socket\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.103356 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.103307 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5858d840-2308-4cf2-b7f3-9f73509bb196-istiod-ca-cert\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.104895 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.104870 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-envoy\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.105093 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.105075 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-podinfo\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.123102 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.123067 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-token\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.123266 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.123236 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlc4r\" (UniqueName: \"kubernetes.io/projected/5858d840-2308-4cf2-b7f3-9f73509bb196-kube-api-access-nlc4r\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.170741 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.170671 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m"] Nov 04 03:58:36.173928 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:58:36.173892 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf15a8b2c_4f91_4792_869d_26e339189978.slice/crio-e0631559dde7614682e079e86346a356cb5ecf776a25ff8a1b12ad481f03cccd WatchSource:0}: Error finding container e0631559dde7614682e079e86346a356cb5ecf776a25ff8a1b12ad481f03cccd: Status 404 returned error can't find the container with id e0631559dde7614682e079e86346a356cb5ecf776a25ff8a1b12ad481f03cccd Nov 04 03:58:36.176808 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.176724 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:58:36.176937 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.176873 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:58:36.176937 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.176915 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:58:36.240399 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.240360 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:36.544168 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.544124 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w"] Nov 04 03:58:36.559102 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.559052 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:58:36.559256 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.559157 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:58:36.559256 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.559200 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:58:36.920538 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.920430 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" event={"ID":"f15a8b2c-4f91-4792-869d-26e339189978","Type":"ContainerStarted","Data":"1052f1e70a12d7d36d17cbef0195b13f379fba4394bbe1e7fde7d4bda4ef6b51"} Nov 04 03:58:36.920538 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.920505 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" event={"ID":"f15a8b2c-4f91-4792-869d-26e339189978","Type":"ContainerStarted","Data":"e0631559dde7614682e079e86346a356cb5ecf776a25ff8a1b12ad481f03cccd"} Nov 04 03:58:36.922744 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.922706 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" event={"ID":"5858d840-2308-4cf2-b7f3-9f73509bb196","Type":"ContainerStarted","Data":"0701a9639a9b39871739c0fc3256eef7bec20e71d2fcc43068b235bfab0ddece"} Nov 04 03:58:36.922956 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.922800 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" event={"ID":"5858d840-2308-4cf2-b7f3-9f73509bb196","Type":"ContainerStarted","Data":"4fc3c2eb3f2c1d65d957904aa75a50b194fc929317d96e40755089603a494ea6"} Nov 04 03:58:36.946924 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.946839 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" podStartSLOduration=1.94682281 podStartE2EDuration="1.94682281s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:58:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:58:36.945882872 +0000 UTC m=+514.984322838" watchObservedRunningTime="2026-11-04 03:58:36.94682281 +0000 UTC m=+514.985262768" Nov 04 03:58:36.973906 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:36.973840 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" podStartSLOduration=1.9738143830000001 podStartE2EDuration="1.973814383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:58:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:58:36.971945124 +0000 UTC m=+515.010385087" watchObservedRunningTime="2026-11-04 03:58:36.973814383 +0000 UTC m=+515.012254349" Nov 04 03:58:37.241000 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:37.240895 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:37.246514 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:37.246486 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:37.814346 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:37.814288 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:37.820265 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:37.820220 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:37.929527 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:37.929476 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:37.929729 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:37.929544 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:37.930526 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:37.930497 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:58:37.930653 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:37.930585 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:58:42.410966 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:42.410933 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l"] Nov 04 03:58:42.414556 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:42.414531 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l" Nov 04 03:58:42.449880 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:42.449845 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l"] Nov 04 03:58:42.463898 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:42.463861 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w5-56a44f-dockercfg-7m7xn\"" Nov 04 03:58:42.474932 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:42.474898 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcp78\" (UniqueName: \"kubernetes.io/projected/31648502-f58d-43a9-a746-bcfb74eb70f7-kube-api-access-xcp78\") pod \"payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l\" (UID: \"31648502-f58d-43a9-a746-bcfb74eb70f7\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l" Nov 04 03:58:42.475080 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:42.475030 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31648502-f58d-43a9-a746-bcfb74eb70f7-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l\" (UID: \"31648502-f58d-43a9-a746-bcfb74eb70f7\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l" Nov 04 03:58:42.507598 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:42.507563 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w5-56a44f\"" Nov 04 03:58:42.575615 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:42.575577 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcp78\" (UniqueName: \"kubernetes.io/projected/31648502-f58d-43a9-a746-bcfb74eb70f7-kube-api-access-xcp78\") pod \"payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l\" (UID: \"31648502-f58d-43a9-a746-bcfb74eb70f7\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l" Nov 04 03:58:42.575795 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:42.575660 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31648502-f58d-43a9-a746-bcfb74eb70f7-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l\" (UID: \"31648502-f58d-43a9-a746-bcfb74eb70f7\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l" Nov 04 03:58:42.576397 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:42.576377 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31648502-f58d-43a9-a746-bcfb74eb70f7-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l\" (UID: \"31648502-f58d-43a9-a746-bcfb74eb70f7\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l" Nov 04 03:58:42.588432 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:42.588403 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcp78\" (UniqueName: \"kubernetes.io/projected/31648502-f58d-43a9-a746-bcfb74eb70f7-kube-api-access-xcp78\") pod \"payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l\" (UID: \"31648502-f58d-43a9-a746-bcfb74eb70f7\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l" Nov 04 03:58:42.729665 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:42.729562 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l" Nov 04 03:58:43.021482 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:43.016534 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l"] Nov 04 03:58:43.027643 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:58:43.027601 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31648502_f58d_43a9_a746_bcfb74eb70f7.slice/crio-7761f255fa61a255e285d6f5d35bbced486d8b1a2da5c96b4db19729d09bde33 WatchSource:0}: Error finding container 7761f255fa61a255e285d6f5d35bbced486d8b1a2da5c96b4db19729d09bde33: Status 404 returned error can't find the container with id 7761f255fa61a255e285d6f5d35bbced486d8b1a2da5c96b4db19729d09bde33 Nov 04 03:58:43.029670 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:43.029650 6000 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 03:58:43.267397 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:43.267364 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s"] Nov 04 03:58:43.272254 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:43.272192 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s" Nov 04 03:58:43.305304 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:43.305239 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w6-6a9cdb-dockercfg-twtnr\"" Nov 04 03:58:43.307606 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:43.307575 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w6-6a9cdb\"" Nov 04 03:58:43.309451 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:43.309420 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s"] Nov 04 03:58:43.394433 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:43.391949 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52e771b7-2c73-4b2f-8266-f37cb72490bd-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s\" (UID: \"52e771b7-2c73-4b2f-8266-f37cb72490bd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s" Nov 04 03:58:43.394433 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:43.392251 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z2vh\" (UniqueName: \"kubernetes.io/projected/52e771b7-2c73-4b2f-8266-f37cb72490bd-kube-api-access-9z2vh\") pod \"payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s\" (UID: \"52e771b7-2c73-4b2f-8266-f37cb72490bd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s" Nov 04 03:58:43.495028 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:43.494977 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52e771b7-2c73-4b2f-8266-f37cb72490bd-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s\" (UID: \"52e771b7-2c73-4b2f-8266-f37cb72490bd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s" Nov 04 03:58:43.495028 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:43.495037 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z2vh\" (UniqueName: \"kubernetes.io/projected/52e771b7-2c73-4b2f-8266-f37cb72490bd-kube-api-access-9z2vh\") pod \"payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s\" (UID: \"52e771b7-2c73-4b2f-8266-f37cb72490bd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s" Nov 04 03:58:43.495706 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:43.495683 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52e771b7-2c73-4b2f-8266-f37cb72490bd-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s\" (UID: \"52e771b7-2c73-4b2f-8266-f37cb72490bd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s" Nov 04 03:58:43.524608 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:43.524526 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z2vh\" (UniqueName: \"kubernetes.io/projected/52e771b7-2c73-4b2f-8266-f37cb72490bd-kube-api-access-9z2vh\") pod \"payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s\" (UID: \"52e771b7-2c73-4b2f-8266-f37cb72490bd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s" Nov 04 03:58:43.585887 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:43.585850 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s" Nov 04 03:58:43.880204 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:43.880157 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s"] Nov 04 03:58:43.882318 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:58:43.882285 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52e771b7_2c73_4b2f_8266_f37cb72490bd.slice/crio-a6799e341eda5e4baab19d65c3a15af1e96c6cf7ae950e6cdb277a5d28f16e5c WatchSource:0}: Error finding container a6799e341eda5e4baab19d65c3a15af1e96c6cf7ae950e6cdb277a5d28f16e5c: Status 404 returned error can't find the container with id a6799e341eda5e4baab19d65c3a15af1e96c6cf7ae950e6cdb277a5d28f16e5c Nov 04 03:58:43.963357 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:43.963316 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l" event={"ID":"31648502-f58d-43a9-a746-bcfb74eb70f7","Type":"ContainerStarted","Data":"7761f255fa61a255e285d6f5d35bbced486d8b1a2da5c96b4db19729d09bde33"} Nov 04 03:58:43.964641 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:43.964576 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s" event={"ID":"52e771b7-2c73-4b2f-8266-f37cb72490bd","Type":"ContainerStarted","Data":"a6799e341eda5e4baab19d65c3a15af1e96c6cf7ae950e6cdb277a5d28f16e5c"} Nov 04 03:58:44.822665 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:44.822635 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-zf6m4"] Nov 04 03:58:45.562985 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.562948 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd"] Nov 04 03:58:45.571398 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.571365 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd"] Nov 04 03:58:45.571545 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.571500 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd" Nov 04 03:58:45.576083 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.576010 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r"] Nov 04 03:58:45.578769 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.577148 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-64c20d\"" Nov 04 03:58:45.578769 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.577372 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-64c20d-dockercfg-xxlzx\"" Nov 04 03:58:45.580244 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.580217 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r" Nov 04 03:58:45.590738 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.590327 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r"] Nov 04 03:58:45.630183 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.629901 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv8lp\" (UniqueName: \"kubernetes.io/projected/e48d6395-918f-4ebd-91c9-4b93edbb520d-kube-api-access-gv8lp\") pod \"payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r\" (UID: \"e48d6395-918f-4ebd-91c9-4b93edbb520d\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r" Nov 04 03:58:45.630183 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.629959 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs89s\" (UniqueName: \"kubernetes.io/projected/200578c9-4fac-445d-b697-55d8d79310fb-kube-api-access-zs89s\") pod \"payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd\" (UID: \"200578c9-4fac-445d-b697-55d8d79310fb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd" Nov 04 03:58:45.630183 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.630015 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/200578c9-4fac-445d-b697-55d8d79310fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd\" (UID: \"200578c9-4fac-445d-b697-55d8d79310fb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd" Nov 04 03:58:45.630183 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.630047 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e48d6395-918f-4ebd-91c9-4b93edbb520d-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r\" (UID: \"e48d6395-918f-4ebd-91c9-4b93edbb520d\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r" Nov 04 03:58:45.761809 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.758276 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv8lp\" (UniqueName: \"kubernetes.io/projected/e48d6395-918f-4ebd-91c9-4b93edbb520d-kube-api-access-gv8lp\") pod \"payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r\" (UID: \"e48d6395-918f-4ebd-91c9-4b93edbb520d\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r" Nov 04 03:58:45.761809 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.758333 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs89s\" (UniqueName: \"kubernetes.io/projected/200578c9-4fac-445d-b697-55d8d79310fb-kube-api-access-zs89s\") pod \"payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd\" (UID: \"200578c9-4fac-445d-b697-55d8d79310fb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd" Nov 04 03:58:45.761809 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.758397 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/200578c9-4fac-445d-b697-55d8d79310fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd\" (UID: \"200578c9-4fac-445d-b697-55d8d79310fb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd" Nov 04 03:58:45.761809 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.758432 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e48d6395-918f-4ebd-91c9-4b93edbb520d-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r\" (UID: \"e48d6395-918f-4ebd-91c9-4b93edbb520d\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r" Nov 04 03:58:45.761809 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.759457 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e48d6395-918f-4ebd-91c9-4b93edbb520d-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r\" (UID: \"e48d6395-918f-4ebd-91c9-4b93edbb520d\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r" Nov 04 03:58:45.761809 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.760702 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/200578c9-4fac-445d-b697-55d8d79310fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd\" (UID: \"200578c9-4fac-445d-b697-55d8d79310fb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd" Nov 04 03:58:45.775928 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.775863 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv8lp\" (UniqueName: \"kubernetes.io/projected/e48d6395-918f-4ebd-91c9-4b93edbb520d-kube-api-access-gv8lp\") pod \"payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r\" (UID: \"e48d6395-918f-4ebd-91c9-4b93edbb520d\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r" Nov 04 03:58:45.784655 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.784607 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs89s\" (UniqueName: \"kubernetes.io/projected/200578c9-4fac-445d-b697-55d8d79310fb-kube-api-access-zs89s\") pod \"payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd\" (UID: \"200578c9-4fac-445d-b697-55d8d79310fb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd" Nov 04 03:58:45.833343 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.831191 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4"] Nov 04 03:58:45.839251 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.837207 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4" Nov 04 03:58:45.848479 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.846697 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4"] Nov 04 03:58:45.849639 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.849583 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bwgrq\"" Nov 04 03:58:45.849639 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.849596 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-6a9cdb-serving-cert\"" Nov 04 03:58:45.849946 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.849736 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Nov 04 03:58:45.895851 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.895818 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd" Nov 04 03:58:45.907623 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.907595 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r" Nov 04 03:58:45.963067 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.963031 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5195f539-2ed2-45c8-bdbe-51d098e321b8-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4\" (UID: \"5195f539-2ed2-45c8-bdbe-51d098e321b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4" Nov 04 03:58:45.963211 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.963080 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5195f539-2ed2-45c8-bdbe-51d098e321b8-metrics-certs\") pod \"maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4\" (UID: \"5195f539-2ed2-45c8-bdbe-51d098e321b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4" Nov 04 03:58:45.963211 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.963180 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5195f539-2ed2-45c8-bdbe-51d098e321b8-maas-api-tls\") pod \"maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4\" (UID: \"5195f539-2ed2-45c8-bdbe-51d098e321b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4" Nov 04 03:58:45.963296 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:45.963282 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsg57\" (UniqueName: \"kubernetes.io/projected/5195f539-2ed2-45c8-bdbe-51d098e321b8-kube-api-access-dsg57\") pod \"maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4\" (UID: \"5195f539-2ed2-45c8-bdbe-51d098e321b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4" Nov 04 03:58:46.073798 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:46.070307 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5195f539-2ed2-45c8-bdbe-51d098e321b8-maas-api-tls\") pod \"maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4\" (UID: \"5195f539-2ed2-45c8-bdbe-51d098e321b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4" Nov 04 03:58:46.073798 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:46.070427 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsg57\" (UniqueName: \"kubernetes.io/projected/5195f539-2ed2-45c8-bdbe-51d098e321b8-kube-api-access-dsg57\") pod \"maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4\" (UID: \"5195f539-2ed2-45c8-bdbe-51d098e321b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4" Nov 04 03:58:46.073798 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:46.070479 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5195f539-2ed2-45c8-bdbe-51d098e321b8-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4\" (UID: \"5195f539-2ed2-45c8-bdbe-51d098e321b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4" Nov 04 03:58:46.073798 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:46.070506 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5195f539-2ed2-45c8-bdbe-51d098e321b8-metrics-certs\") pod \"maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4\" (UID: \"5195f539-2ed2-45c8-bdbe-51d098e321b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4" Nov 04 03:58:46.085940 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:46.084306 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5195f539-2ed2-45c8-bdbe-51d098e321b8-maas-api-tls\") pod \"maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4\" (UID: \"5195f539-2ed2-45c8-bdbe-51d098e321b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4" Nov 04 03:58:46.097787 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:46.097091 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsg57\" (UniqueName: \"kubernetes.io/projected/5195f539-2ed2-45c8-bdbe-51d098e321b8-kube-api-access-dsg57\") pod \"maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4\" (UID: \"5195f539-2ed2-45c8-bdbe-51d098e321b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4" Nov 04 03:58:46.104825 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:46.104748 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5195f539-2ed2-45c8-bdbe-51d098e321b8-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4\" (UID: \"5195f539-2ed2-45c8-bdbe-51d098e321b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4" Nov 04 03:58:46.107426 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:46.107365 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5195f539-2ed2-45c8-bdbe-51d098e321b8-metrics-certs\") pod \"maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4\" (UID: \"5195f539-2ed2-45c8-bdbe-51d098e321b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4" Nov 04 03:58:46.173581 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:46.173537 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4" Nov 04 03:58:46.205931 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:46.205733 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd"] Nov 04 03:58:46.213813 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:58:46.213782 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod200578c9_4fac_445d_b697_55d8d79310fb.slice/crio-259867f16c126adcc0441ca1307b4b9150dadb668f2523667116436fd14074c0 WatchSource:0}: Error finding container 259867f16c126adcc0441ca1307b4b9150dadb668f2523667116436fd14074c0: Status 404 returned error can't find the container with id 259867f16c126adcc0441ca1307b4b9150dadb668f2523667116436fd14074c0 Nov 04 03:58:46.252273 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:46.244057 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r"] Nov 04 03:58:46.490254 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:46.488621 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4"] Nov 04 03:58:46.495164 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:58:46.495116 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5195f539_2ed2_45c8_bdbe_51d098e321b8.slice/crio-41279bb6bda46d9c502cf7f4857d9ad4e47a36e2ce08399875e6928067915ae8 WatchSource:0}: Error finding container 41279bb6bda46d9c502cf7f4857d9ad4e47a36e2ce08399875e6928067915ae8: Status 404 returned error can't find the container with id 41279bb6bda46d9c502cf7f4857d9ad4e47a36e2ce08399875e6928067915ae8 Nov 04 03:58:46.989023 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:46.988990 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r" event={"ID":"e48d6395-918f-4ebd-91c9-4b93edbb520d","Type":"ContainerStarted","Data":"372e9783aff79dc67f2838732ec1a84b0c5a0509e771c12e06aca755e011446e"} Nov 04 03:58:46.989415 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:46.989060 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r" event={"ID":"e48d6395-918f-4ebd-91c9-4b93edbb520d","Type":"ContainerStarted","Data":"e1323e2ed1718f262e315bc13d1f8faa32e0c1a24a0a11f0a73360117effc811"} Nov 04 03:58:46.989415 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:46.989186 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r" Nov 04 03:58:46.990993 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:46.990958 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4" event={"ID":"5195f539-2ed2-45c8-bdbe-51d098e321b8","Type":"ContainerStarted","Data":"41279bb6bda46d9c502cf7f4857d9ad4e47a36e2ce08399875e6928067915ae8"} Nov 04 03:58:46.993050 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:46.993018 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l" event={"ID":"31648502-f58d-43a9-a746-bcfb74eb70f7","Type":"ContainerStarted","Data":"225646120e3d005645cd4800c05f154e4909d80f60de719df674448ad3021f91"} Nov 04 03:58:46.993270 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:46.993247 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l" Nov 04 03:58:46.994916 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:46.994890 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd" event={"ID":"200578c9-4fac-445d-b697-55d8d79310fb","Type":"ContainerStarted","Data":"0064031ada67f2e95496b8d794f38f84a17c5064ce56c02fa44a64d5025fb1c3"} Nov 04 03:58:46.995014 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:46.994940 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd" event={"ID":"200578c9-4fac-445d-b697-55d8d79310fb","Type":"ContainerStarted","Data":"259867f16c126adcc0441ca1307b4b9150dadb668f2523667116436fd14074c0"} Nov 04 03:58:46.995150 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:46.995133 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd" Nov 04 03:58:47.009507 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:47.009450 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r" podStartSLOduration=2.009431641 podStartE2EDuration="2.009431641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:58:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:58:47.006972162 +0000 UTC m=+525.045412129" watchObservedRunningTime="2026-11-04 03:58:47.009431641 +0000 UTC m=+525.047871606" Nov 04 03:58:47.027134 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:47.027084 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd" podStartSLOduration=2.027067022 podStartE2EDuration="2.027067022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:58:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:58:47.026370506 +0000 UTC m=+525.064810471" watchObservedRunningTime="2026-11-04 03:58:47.027067022 +0000 UTC m=+525.065506987" Nov 04 03:58:47.104042 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:47.103986 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l" podStartSLOduration=1.9737948159999998 podStartE2EDuration="5.103971722s" totalImagesPullingTime="3.130176906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:58:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:58:47.099574379 +0000 UTC m=+525.138014344" watchObservedRunningTime="2026-11-04 03:58:47.103971722 +0000 UTC m=+525.142411685" Nov 04 03:58:48.003704 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:48.003668 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4" event={"ID":"5195f539-2ed2-45c8-bdbe-51d098e321b8","Type":"ContainerStarted","Data":"874a541b4b75600e526259188c85ef943bf964361a00ad65cbe0811aeec37d14"} Nov 04 03:58:48.004199 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:48.003740 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4" Nov 04 03:58:48.005398 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:48.005369 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s" event={"ID":"52e771b7-2c73-4b2f-8266-f37cb72490bd","Type":"ContainerStarted","Data":"222676571a7b3bc0c14c3ee89cc468fc83b83d000a9a74d70b56d8567a73a7c4"} Nov 04 03:58:48.005867 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:48.005844 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s" Nov 04 03:58:48.027368 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:48.027315 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4" podStartSLOduration=2.6044190350000003 podStartE2EDuration="3.027299431s" totalImagesPullingTime="422.880396ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:58:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:58:48.025194606 +0000 UTC m=+526.063634571" watchObservedRunningTime="2026-11-04 03:58:48.027299431 +0000 UTC m=+526.065739396" Nov 04 03:58:48.046293 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:48.046225 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s" podStartSLOduration=2.011780908 podStartE2EDuration="5.046203466s" totalImagesPullingTime="3.034422558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:58:48.044619211 +0000 UTC m=+526.083059221" watchObservedRunningTime="2026-11-04 03:58:48.046203466 +0000 UTC m=+526.084643432" Nov 04 03:58:48.636533 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:48.636496 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s"] Nov 04 03:58:48.640707 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:48.640677 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s" Nov 04 03:58:48.650627 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:48.649716 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-64c20d-serving-cert\"" Nov 04 03:58:48.653621 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:48.653588 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s"] Nov 04 03:58:48.802593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:48.802547 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97aca100-541e-48ff-9b0e-5d4853b700ff-maas-api-tls\") pod \"maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s\" (UID: \"97aca100-541e-48ff-9b0e-5d4853b700ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s" Nov 04 03:58:48.802593 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:48.802593 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2gv9\" (UniqueName: \"kubernetes.io/projected/97aca100-541e-48ff-9b0e-5d4853b700ff-kube-api-access-d2gv9\") pod \"maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s\" (UID: \"97aca100-541e-48ff-9b0e-5d4853b700ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s" Nov 04 03:58:48.802853 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:48.802679 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97aca100-541e-48ff-9b0e-5d4853b700ff-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s\" (UID: \"97aca100-541e-48ff-9b0e-5d4853b700ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s" Nov 04 03:58:48.802853 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:48.802807 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97aca100-541e-48ff-9b0e-5d4853b700ff-metrics-certs\") pod \"maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s\" (UID: \"97aca100-541e-48ff-9b0e-5d4853b700ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s" Nov 04 03:58:48.903962 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:48.903867 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97aca100-541e-48ff-9b0e-5d4853b700ff-maas-api-tls\") pod \"maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s\" (UID: \"97aca100-541e-48ff-9b0e-5d4853b700ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s" Nov 04 03:58:48.903962 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:48.903906 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2gv9\" (UniqueName: \"kubernetes.io/projected/97aca100-541e-48ff-9b0e-5d4853b700ff-kube-api-access-d2gv9\") pod \"maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s\" (UID: \"97aca100-541e-48ff-9b0e-5d4853b700ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s" Nov 04 03:58:48.903962 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:48.903933 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97aca100-541e-48ff-9b0e-5d4853b700ff-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s\" (UID: \"97aca100-541e-48ff-9b0e-5d4853b700ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s" Nov 04 03:58:48.903962 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:48.903957 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97aca100-541e-48ff-9b0e-5d4853b700ff-metrics-certs\") pod \"maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s\" (UID: \"97aca100-541e-48ff-9b0e-5d4853b700ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s" Nov 04 03:58:48.904775 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:48.904731 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97aca100-541e-48ff-9b0e-5d4853b700ff-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s\" (UID: \"97aca100-541e-48ff-9b0e-5d4853b700ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s" Nov 04 03:58:48.906668 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:48.906643 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97aca100-541e-48ff-9b0e-5d4853b700ff-maas-api-tls\") pod \"maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s\" (UID: \"97aca100-541e-48ff-9b0e-5d4853b700ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s" Nov 04 03:58:48.906825 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:48.906687 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97aca100-541e-48ff-9b0e-5d4853b700ff-metrics-certs\") pod \"maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s\" (UID: \"97aca100-541e-48ff-9b0e-5d4853b700ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s" Nov 04 03:58:48.916303 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:48.916272 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2gv9\" (UniqueName: \"kubernetes.io/projected/97aca100-541e-48ff-9b0e-5d4853b700ff-kube-api-access-d2gv9\") pod \"maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s\" (UID: \"97aca100-541e-48ff-9b0e-5d4853b700ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s" Nov 04 03:58:48.960175 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:48.960137 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s" Nov 04 03:58:49.251355 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:49.251327 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s"] Nov 04 03:58:49.254088 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:58:49.254050 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97aca100_541e_48ff_9b0e_5d4853b700ff.slice/crio-e881be6f353687fbe6c2c9e4e99c044eba88770acc35ba41f2db1d456b8610c2 WatchSource:0}: Error finding container e881be6f353687fbe6c2c9e4e99c044eba88770acc35ba41f2db1d456b8610c2: Status 404 returned error can't find the container with id e881be6f353687fbe6c2c9e4e99c044eba88770acc35ba41f2db1d456b8610c2 Nov 04 03:58:49.708540 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:49.708508 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-zf6m4"] Nov 04 03:58:50.020614 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:50.020524 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s" event={"ID":"97aca100-541e-48ff-9b0e-5d4853b700ff","Type":"ContainerStarted","Data":"efa88804b1b545a3dcdb5e1918bd7bf7383e5b2a5538fadd6cfcd9b0feafbec0"} Nov 04 03:58:50.020614 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:50.020589 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s" Nov 04 03:58:50.020860 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:50.020615 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s" event={"ID":"97aca100-541e-48ff-9b0e-5d4853b700ff","Type":"ContainerStarted","Data":"e881be6f353687fbe6c2c9e4e99c044eba88770acc35ba41f2db1d456b8610c2"} Nov 04 03:58:50.048559 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:50.048499 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s" podStartSLOduration=1.614712451 podStartE2EDuration="2.048479023s" totalImagesPullingTime="433.766572ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:58:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:58:50.047637298 +0000 UTC m=+528.086077314" watchObservedRunningTime="2026-11-04 03:58:50.048479023 +0000 UTC m=+528.086919028" Nov 04 03:58:50.804429 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:58:50.804388 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 03:58:54.045992 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:54.045953 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4" Nov 04 03:58:56.057174 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:56.057034 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s" Nov 04 03:58:56.162424 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:56.162386 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-zf6m4"] Nov 04 03:58:56.838680 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:56.836680 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr"] Nov 04 03:58:56.844396 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:56.842135 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 03:58:56.852141 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:56.852100 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-56a44f\"/\"openshift-service-ca.crt\"" Nov 04 03:58:56.852141 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:56.853661 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-56a44f\"/\"simulator-w5-56a44f-kserve-self-signed-certs\"" Nov 04 03:58:56.852141 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:56.853718 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-56a44f\"/\"default-dockercfg-vfh6g\"" Nov 04 03:58:56.854079 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:56.853800 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-56a44f\"/\"kube-root-ca.crt\"" Nov 04 03:58:56.860019 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:56.856874 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr"] Nov 04 03:58:56.941741 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:56.941698 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f996ad18-5c32-4945-822e-62ed92214a54-tls-certs\") pod \"simulator-w5-56a44f-kserve-779787b884-jhjdr\" (UID: \"f996ad18-5c32-4945-822e-62ed92214a54\") " pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 03:58:56.941971 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:56.941804 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-home\") pod \"simulator-w5-56a44f-kserve-779787b884-jhjdr\" (UID: \"f996ad18-5c32-4945-822e-62ed92214a54\") " pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 03:58:56.941971 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:56.941857 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-model-cache\") pod \"simulator-w5-56a44f-kserve-779787b884-jhjdr\" (UID: \"f996ad18-5c32-4945-822e-62ed92214a54\") " pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 03:58:56.941971 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:56.941884 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-dshm\") pod \"simulator-w5-56a44f-kserve-779787b884-jhjdr\" (UID: \"f996ad18-5c32-4945-822e-62ed92214a54\") " pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 03:58:56.942163 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:56.941996 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvv2h\" (UniqueName: \"kubernetes.io/projected/f996ad18-5c32-4945-822e-62ed92214a54-kube-api-access-lvv2h\") pod \"simulator-w5-56a44f-kserve-779787b884-jhjdr\" (UID: \"f996ad18-5c32-4945-822e-62ed92214a54\") " pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 03:58:56.942163 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:56.942098 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-tmp-dir\") pod \"simulator-w5-56a44f-kserve-779787b884-jhjdr\" (UID: \"f996ad18-5c32-4945-822e-62ed92214a54\") " pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 03:58:57.042891 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:57.042853 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-tmp-dir\") pod \"simulator-w5-56a44f-kserve-779787b884-jhjdr\" (UID: \"f996ad18-5c32-4945-822e-62ed92214a54\") " pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 03:58:57.043091 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:57.042966 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f996ad18-5c32-4945-822e-62ed92214a54-tls-certs\") pod \"simulator-w5-56a44f-kserve-779787b884-jhjdr\" (UID: \"f996ad18-5c32-4945-822e-62ed92214a54\") " pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 03:58:57.043091 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:57.043021 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-home\") pod \"simulator-w5-56a44f-kserve-779787b884-jhjdr\" (UID: \"f996ad18-5c32-4945-822e-62ed92214a54\") " pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 03:58:57.043091 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:57.043067 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-model-cache\") pod \"simulator-w5-56a44f-kserve-779787b884-jhjdr\" (UID: \"f996ad18-5c32-4945-822e-62ed92214a54\") " pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 03:58:57.043270 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:57.043096 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-dshm\") pod \"simulator-w5-56a44f-kserve-779787b884-jhjdr\" (UID: \"f996ad18-5c32-4945-822e-62ed92214a54\") " pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 03:58:57.043270 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:57.043130 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvv2h\" (UniqueName: \"kubernetes.io/projected/f996ad18-5c32-4945-822e-62ed92214a54-kube-api-access-lvv2h\") pod \"simulator-w5-56a44f-kserve-779787b884-jhjdr\" (UID: \"f996ad18-5c32-4945-822e-62ed92214a54\") " pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 03:58:57.043374 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:57.043341 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-tmp-dir\") pod \"simulator-w5-56a44f-kserve-779787b884-jhjdr\" (UID: \"f996ad18-5c32-4945-822e-62ed92214a54\") " pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 03:58:57.043693 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:57.043566 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-model-cache\") pod \"simulator-w5-56a44f-kserve-779787b884-jhjdr\" (UID: \"f996ad18-5c32-4945-822e-62ed92214a54\") " pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 03:58:57.043844 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:57.043737 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-home\") pod \"simulator-w5-56a44f-kserve-779787b884-jhjdr\" (UID: \"f996ad18-5c32-4945-822e-62ed92214a54\") " pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 03:58:57.047990 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:57.047965 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-dshm\") pod \"simulator-w5-56a44f-kserve-779787b884-jhjdr\" (UID: \"f996ad18-5c32-4945-822e-62ed92214a54\") " pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 03:58:57.048323 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:57.048301 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f996ad18-5c32-4945-822e-62ed92214a54-tls-certs\") pod \"simulator-w5-56a44f-kserve-779787b884-jhjdr\" (UID: \"f996ad18-5c32-4945-822e-62ed92214a54\") " pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 03:58:57.056010 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:57.055972 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvv2h\" (UniqueName: \"kubernetes.io/projected/f996ad18-5c32-4945-822e-62ed92214a54-kube-api-access-lvv2h\") pod \"simulator-w5-56a44f-kserve-779787b884-jhjdr\" (UID: \"f996ad18-5c32-4945-822e-62ed92214a54\") " pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 03:58:57.175527 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:57.175438 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 03:58:57.480828 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:57.480701 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr"] Nov 04 03:58:58.009656 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:58.009624 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd" Nov 04 03:58:58.012956 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:58.012902 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l" Nov 04 03:58:58.014740 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:58.014714 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r" Nov 04 03:58:58.131774 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:58:58.131665 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" event={"ID":"f996ad18-5c32-4945-822e-62ed92214a54","Type":"ContainerStarted","Data":"2cb80b764341d73ca3f1b34319f318e4fd493a5a307bd9669f061423a983e6c4"} Nov 04 03:59:00.023865 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:00.023834 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s" Nov 04 03:59:00.147588 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:00.147543 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" event={"ID":"f996ad18-5c32-4945-822e-62ed92214a54","Type":"ContainerStarted","Data":"bb10a017c34e6d7847ea8accc778bae6f2832eadd493af8d23081e8d81bd6a0e"} Nov 04 03:59:00.149362 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:00.149330 6000 generic.go:356] "Generic (PLEG): container finished" podID="f15a8b2c-4f91-4792-869d-26e339189978" containerID="1052f1e70a12d7d36d17cbef0195b13f379fba4394bbe1e7fde7d4bda4ef6b51" exitCode=0 Nov 04 03:59:00.149509 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:00.149408 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" event={"ID":"f15a8b2c-4f91-4792-869d-26e339189978","Type":"ContainerDied","Data":"1052f1e70a12d7d36d17cbef0195b13f379fba4394bbe1e7fde7d4bda4ef6b51"} Nov 04 03:59:00.149943 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:00.149926 6000 scope.go:122] "RemoveContainer" containerID="1052f1e70a12d7d36d17cbef0195b13f379fba4394bbe1e7fde7d4bda4ef6b51" Nov 04 03:59:00.151258 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:00.151229 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:59:00.151345 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:00.151296 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:59:00.151384 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:00.151357 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:59:00.174150 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:00.174104 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" podStartSLOduration=2.444145582 podStartE2EDuration="4.174088655s" totalImagesPullingTime="1.729943073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:58:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:59:00.172540599 +0000 UTC m=+538.210980564" watchObservedRunningTime="2026-11-04 03:59:00.174088655 +0000 UTC m=+538.212528648" Nov 04 03:59:00.176134 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:00.176113 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 03:59:01.158202 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:01.158161 6000 generic.go:356] "Generic (PLEG): container finished" podID="5858d840-2308-4cf2-b7f3-9f73509bb196" containerID="0701a9639a9b39871739c0fc3256eef7bec20e71d2fcc43068b235bfab0ddece" exitCode=0 Nov 04 03:59:01.158712 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:01.158237 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" event={"ID":"5858d840-2308-4cf2-b7f3-9f73509bb196","Type":"ContainerDied","Data":"0701a9639a9b39871739c0fc3256eef7bec20e71d2fcc43068b235bfab0ddece"} Nov 04 03:59:01.158712 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:01.158701 6000 scope.go:122] "RemoveContainer" containerID="0701a9639a9b39871739c0fc3256eef7bec20e71d2fcc43068b235bfab0ddece" Nov 04 03:59:01.162019 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:01.160155 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:59:01.162019 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:01.160226 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:59:01.162019 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:01.160285 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:59:01.162019 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:01.160779 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" event={"ID":"f15a8b2c-4f91-4792-869d-26e339189978","Type":"ContainerStarted","Data":"22407c01e6b73c77b5e3523d310725fb5f996d5ddb360a572cda1a492f94f20d"} Nov 04 03:59:01.816670 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:01.816205 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:59:01.816670 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:01.816341 6000 patch_prober.go:29] interesting pod/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Nov 04 03:59:01.816670 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:01.816387 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" podUID="f15a8b2c-4f91-4792-869d-26e339189978" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Nov 04 03:59:02.199896 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:02.199816 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" event={"ID":"5858d840-2308-4cf2-b7f3-9f73509bb196","Type":"ContainerStarted","Data":"4cd1001714fcfa5933cecb2b03775c1cd5dca827e8996ffaabfd0f66e68ad36f"} Nov 04 03:59:02.242782 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:02.242187 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:59:02.242782 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:02.242316 6000 patch_prober.go:29] interesting pod/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 04 03:59:02.242782 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:02.242358 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" podUID="5858d840-2308-4cf2-b7f3-9f73509bb196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 04 03:59:02.820793 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:02.820736 6000 patch_prober.go:29] interesting pod/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Nov 04 03:59:02.821114 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:02.821076 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" podUID="f15a8b2c-4f91-4792-869d-26e339189978" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Nov 04 03:59:03.244312 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:03.244205 6000 patch_prober.go:29] interesting pod/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 04 03:59:03.244954 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:03.244921 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" podUID="5858d840-2308-4cf2-b7f3-9f73509bb196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 04 03:59:03.827782 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:03.817443 6000 patch_prober.go:29] interesting pod/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Nov 04 03:59:03.827782 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:03.817511 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" podUID="f15a8b2c-4f91-4792-869d-26e339189978" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Nov 04 03:59:04.241330 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:04.241225 6000 patch_prober.go:29] interesting pod/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 04 03:59:04.241679 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:04.241625 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" podUID="5858d840-2308-4cf2-b7f3-9f73509bb196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 04 03:59:04.815268 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:04.815226 6000 patch_prober.go:29] interesting pod/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Nov 04 03:59:04.815872 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:04.815837 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" podUID="f15a8b2c-4f91-4792-869d-26e339189978" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Nov 04 03:59:05.241294 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:05.241196 6000 patch_prober.go:29] interesting pod/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 04 03:59:05.241563 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:05.241531 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" podUID="5858d840-2308-4cf2-b7f3-9f73509bb196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 04 03:59:05.817217 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:05.817175 6000 patch_prober.go:29] interesting pod/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Nov 04 03:59:05.817853 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:05.817807 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" podUID="f15a8b2c-4f91-4792-869d-26e339189978" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Nov 04 03:59:06.098835 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:06.096800 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-zf6m4"] Nov 04 03:59:06.251262 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:06.251222 6000 patch_prober.go:29] interesting pod/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 04 03:59:06.251550 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:06.251519 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" podUID="5858d840-2308-4cf2-b7f3-9f73509bb196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 04 03:59:07.148092 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:07.148057 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:59:07.150195 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:07.150172 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:59:07.153207 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:07.153185 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 03:59:07.177552 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:07.177518 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 03:59:07.193884 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:07.193849 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 03:59:07.259784 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:07.259620 6000 patch_prober.go:29] interesting pod/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 04 03:59:07.259784 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:07.259677 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" podUID="5858d840-2308-4cf2-b7f3-9f73509bb196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 04 03:59:07.287474 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:07.286011 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 03:59:07.929603 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:07.929569 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:59:08.246294 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:08.246213 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:59:08.271443 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:08.271403 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 03:59:08.948799 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:08.948749 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh"] Nov 04 03:59:08.954376 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:08.954351 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 03:59:08.963265 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:08.963232 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh"] Nov 04 03:59:08.965039 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:08.965018 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-4afe0e\"/\"default-dockercfg-gqkhp\"" Nov 04 03:59:08.965204 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:08.965093 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-4afe0e\"/\"simulator-w0-4afe0e-kserve-self-signed-certs\"" Nov 04 03:59:09.057540 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:09.057501 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-4afe0e\"/\"openshift-service-ca.crt\"" Nov 04 03:59:09.057540 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:09.057532 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-4afe0e\"/\"kube-root-ca.crt\"" Nov 04 03:59:09.069310 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:09.069270 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-tmp-dir\") pod \"simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh\" (UID: \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\") " pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 03:59:09.069489 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:09.069404 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-home\") pod \"simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh\" (UID: \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\") " pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 03:59:09.069489 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:09.069480 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w54g\" (UniqueName: \"kubernetes.io/projected/785d143f-077d-4ebd-8b1f-1cd3c67aac47-kube-api-access-9w54g\") pod \"simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh\" (UID: \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\") " pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 03:59:09.069601 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:09.069501 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-dshm\") pod \"simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh\" (UID: \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\") " pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 03:59:09.069669 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:09.069646 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/785d143f-077d-4ebd-8b1f-1cd3c67aac47-tls-certs\") pod \"simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh\" (UID: \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\") " pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 03:59:09.069707 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:09.069692 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-model-cache\") pod \"simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh\" (UID: \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\") " pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 03:59:09.170430 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:09.170382 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/785d143f-077d-4ebd-8b1f-1cd3c67aac47-tls-certs\") pod \"simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh\" (UID: \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\") " pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 03:59:09.170651 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:09.170449 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-model-cache\") pod \"simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh\" (UID: \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\") " pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 03:59:09.170651 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:09.170499 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-tmp-dir\") pod \"simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh\" (UID: \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\") " pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 03:59:09.170651 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:09.170611 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-home\") pod \"simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh\" (UID: \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\") " pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 03:59:09.170849 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:09.170748 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w54g\" (UniqueName: \"kubernetes.io/projected/785d143f-077d-4ebd-8b1f-1cd3c67aac47-kube-api-access-9w54g\") pod \"simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh\" (UID: \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\") " pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 03:59:09.170849 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:09.170823 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-dshm\") pod \"simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh\" (UID: \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\") " pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 03:59:09.171603 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:09.171574 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-tmp-dir\") pod \"simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh\" (UID: \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\") " pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 03:59:09.171743 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:09.171586 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-model-cache\") pod \"simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh\" (UID: \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\") " pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 03:59:09.171916 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:09.171892 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-home\") pod \"simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh\" (UID: \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\") " pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 03:59:09.173506 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:09.173482 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-dshm\") pod \"simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh\" (UID: \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\") " pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 03:59:09.173861 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:09.173839 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/785d143f-077d-4ebd-8b1f-1cd3c67aac47-tls-certs\") pod \"simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh\" (UID: \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\") " pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 03:59:09.184823 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:09.184793 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w54g\" (UniqueName: \"kubernetes.io/projected/785d143f-077d-4ebd-8b1f-1cd3c67aac47-kube-api-access-9w54g\") pod \"simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh\" (UID: \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\") " pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 03:59:09.269429 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:09.269345 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 03:59:09.549656 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:09.549627 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh"] Nov 04 03:59:09.551941 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:59:09.551913 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod785d143f_077d_4ebd_8b1f_1cd3c67aac47.slice/crio-e6818145ded7218693a752031fc0eeadc04eabbbd995a51d575b24d0f0c11621 WatchSource:0}: Error finding container e6818145ded7218693a752031fc0eeadc04eabbbd995a51d575b24d0f0c11621: Status 404 returned error can't find the container with id e6818145ded7218693a752031fc0eeadc04eabbbd995a51d575b24d0f0c11621 Nov 04 03:59:10.284937 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:10.284851 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" event={"ID":"785d143f-077d-4ebd-8b1f-1cd3c67aac47","Type":"ContainerStarted","Data":"f65be9cef0aa4ca34fce40cbda1fc4810c525f886995a7e75c9e3182a145e3bb"} Nov 04 03:59:10.284937 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:10.284895 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" event={"ID":"785d143f-077d-4ebd-8b1f-1cd3c67aac47","Type":"ContainerStarted","Data":"e6818145ded7218693a752031fc0eeadc04eabbbd995a51d575b24d0f0c11621"} Nov 04 03:59:10.306072 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:10.306022 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" podStartSLOduration=2.306008585 podStartE2EDuration="2.306008585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:59:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:59:10.304149554 +0000 UTC m=+548.342589518" watchObservedRunningTime="2026-11-04 03:59:10.306008585 +0000 UTC m=+548.344448749" Nov 04 03:59:11.270058 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:11.270022 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 03:59:14.315307 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.315273 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65"] Nov 04 03:59:14.317911 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.317889 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 03:59:14.328158 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.328127 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-78ca37\"/\"kube-root-ca.crt\"" Nov 04 03:59:14.328158 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.328130 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-78ca37\"/\"openshift-service-ca.crt\"" Nov 04 03:59:14.329314 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.329289 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-78ca37\"/\"simulator-w2-78ca37-kserve-self-signed-certs\"" Nov 04 03:59:14.329472 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.329314 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-78ca37\"/\"default-dockercfg-msrpj\"" Nov 04 03:59:14.337109 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.337082 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65"] Nov 04 03:59:14.418147 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.418105 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-dshm\") pod \"simulator-w2-78ca37-kserve-5fd8887974-vxv65\" (UID: \"9db4519d-0989-45cc-9198-22f41c15c951\") " pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 03:59:14.418364 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.418259 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-tmp-dir\") pod \"simulator-w2-78ca37-kserve-5fd8887974-vxv65\" (UID: \"9db4519d-0989-45cc-9198-22f41c15c951\") " pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 03:59:14.418364 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.418333 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9db4519d-0989-45cc-9198-22f41c15c951-tls-certs\") pod \"simulator-w2-78ca37-kserve-5fd8887974-vxv65\" (UID: \"9db4519d-0989-45cc-9198-22f41c15c951\") " pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 03:59:14.418472 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.418408 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-model-cache\") pod \"simulator-w2-78ca37-kserve-5fd8887974-vxv65\" (UID: \"9db4519d-0989-45cc-9198-22f41c15c951\") " pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 03:59:14.418525 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.418483 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7vlp\" (UniqueName: \"kubernetes.io/projected/9db4519d-0989-45cc-9198-22f41c15c951-kube-api-access-h7vlp\") pod \"simulator-w2-78ca37-kserve-5fd8887974-vxv65\" (UID: \"9db4519d-0989-45cc-9198-22f41c15c951\") " pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 03:59:14.418525 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.418515 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-home\") pod \"simulator-w2-78ca37-kserve-5fd8887974-vxv65\" (UID: \"9db4519d-0989-45cc-9198-22f41c15c951\") " pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 03:59:14.519272 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.519234 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7vlp\" (UniqueName: \"kubernetes.io/projected/9db4519d-0989-45cc-9198-22f41c15c951-kube-api-access-h7vlp\") pod \"simulator-w2-78ca37-kserve-5fd8887974-vxv65\" (UID: \"9db4519d-0989-45cc-9198-22f41c15c951\") " pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 03:59:14.519272 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.519281 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-home\") pod \"simulator-w2-78ca37-kserve-5fd8887974-vxv65\" (UID: \"9db4519d-0989-45cc-9198-22f41c15c951\") " pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 03:59:14.519543 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.519337 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-dshm\") pod \"simulator-w2-78ca37-kserve-5fd8887974-vxv65\" (UID: \"9db4519d-0989-45cc-9198-22f41c15c951\") " pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 03:59:14.519543 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.519392 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-tmp-dir\") pod \"simulator-w2-78ca37-kserve-5fd8887974-vxv65\" (UID: \"9db4519d-0989-45cc-9198-22f41c15c951\") " pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 03:59:14.519543 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.519420 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9db4519d-0989-45cc-9198-22f41c15c951-tls-certs\") pod \"simulator-w2-78ca37-kserve-5fd8887974-vxv65\" (UID: \"9db4519d-0989-45cc-9198-22f41c15c951\") " pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 03:59:14.519543 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.519453 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-model-cache\") pod \"simulator-w2-78ca37-kserve-5fd8887974-vxv65\" (UID: \"9db4519d-0989-45cc-9198-22f41c15c951\") " pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 03:59:14.519861 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.519839 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-home\") pod \"simulator-w2-78ca37-kserve-5fd8887974-vxv65\" (UID: \"9db4519d-0989-45cc-9198-22f41c15c951\") " pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 03:59:14.519861 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.519850 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-model-cache\") pod \"simulator-w2-78ca37-kserve-5fd8887974-vxv65\" (UID: \"9db4519d-0989-45cc-9198-22f41c15c951\") " pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 03:59:14.519990 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.519972 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-tmp-dir\") pod \"simulator-w2-78ca37-kserve-5fd8887974-vxv65\" (UID: \"9db4519d-0989-45cc-9198-22f41c15c951\") " pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 03:59:14.522717 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.522687 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-dshm\") pod \"simulator-w2-78ca37-kserve-5fd8887974-vxv65\" (UID: \"9db4519d-0989-45cc-9198-22f41c15c951\") " pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 03:59:14.523221 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.523199 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9db4519d-0989-45cc-9198-22f41c15c951-tls-certs\") pod \"simulator-w2-78ca37-kserve-5fd8887974-vxv65\" (UID: \"9db4519d-0989-45cc-9198-22f41c15c951\") " pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 03:59:14.537407 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.537378 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7vlp\" (UniqueName: \"kubernetes.io/projected/9db4519d-0989-45cc-9198-22f41c15c951-kube-api-access-h7vlp\") pod \"simulator-w2-78ca37-kserve-5fd8887974-vxv65\" (UID: \"9db4519d-0989-45cc-9198-22f41c15c951\") " pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 03:59:14.639676 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.639586 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 03:59:14.926321 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:14.926289 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65"] Nov 04 03:59:14.928321 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:59:14.928288 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9db4519d_0989_45cc_9198_22f41c15c951.slice/crio-115fdc52b020ad77551f7a1be2a080a55e49f3f5a1acbc17eb656a1161b59316 WatchSource:0}: Error finding container 115fdc52b020ad77551f7a1be2a080a55e49f3f5a1acbc17eb656a1161b59316: Status 404 returned error can't find the container with id 115fdc52b020ad77551f7a1be2a080a55e49f3f5a1acbc17eb656a1161b59316 Nov 04 03:59:15.317309 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:15.317270 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" event={"ID":"9db4519d-0989-45cc-9198-22f41c15c951","Type":"ContainerStarted","Data":"51f18bdc4ebccfb04caf6e7664cb8b6057eef543e0590d5130765f8f9a4a5a5d"} Nov 04 03:59:15.317844 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:15.317318 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" event={"ID":"9db4519d-0989-45cc-9198-22f41c15c951","Type":"ContainerStarted","Data":"115fdc52b020ad77551f7a1be2a080a55e49f3f5a1acbc17eb656a1161b59316"} Nov 04 03:59:15.339334 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:15.339258 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" podStartSLOduration=1.33924364 podStartE2EDuration="1.33924364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:59:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:59:15.337239238 +0000 UTC m=+553.375679216" watchObservedRunningTime="2026-11-04 03:59:15.33924364 +0000 UTC m=+553.377683603" Nov 04 03:59:15.640005 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:15.639906 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 03:59:16.123436 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:16.123401 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-zf6m4"] Nov 04 03:59:19.269737 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:19.269692 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 03:59:19.284733 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:19.284701 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 03:59:19.355804 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:19.355767 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 03:59:20.749798 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:59:20.749731 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 03:59:22.243780 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:22.243520 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg"] Nov 04 03:59:22.260115 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:22.260011 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg"] Nov 04 03:59:22.260292 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:22.260145 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 03:59:22.268790 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:22.268513 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-56a44f\"/\"premium-w5-56a44f-kserve-self-signed-certs\"" Nov 04 03:59:22.301110 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:22.301071 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-tmp-dir\") pod \"premium-w5-56a44f-kserve-b7d69c94f-987lg\" (UID: \"6603cd37-3576-49ff-a87c-e307d4d35706\") " pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 03:59:22.301298 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:22.301134 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-model-cache\") pod \"premium-w5-56a44f-kserve-b7d69c94f-987lg\" (UID: \"6603cd37-3576-49ff-a87c-e307d4d35706\") " pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 03:59:22.301298 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:22.301256 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlqq7\" (UniqueName: \"kubernetes.io/projected/6603cd37-3576-49ff-a87c-e307d4d35706-kube-api-access-hlqq7\") pod \"premium-w5-56a44f-kserve-b7d69c94f-987lg\" (UID: \"6603cd37-3576-49ff-a87c-e307d4d35706\") " pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 03:59:22.301422 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:22.301383 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6603cd37-3576-49ff-a87c-e307d4d35706-tls-certs\") pod \"premium-w5-56a44f-kserve-b7d69c94f-987lg\" (UID: \"6603cd37-3576-49ff-a87c-e307d4d35706\") " pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 03:59:22.301481 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:22.301425 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-dshm\") pod \"premium-w5-56a44f-kserve-b7d69c94f-987lg\" (UID: \"6603cd37-3576-49ff-a87c-e307d4d35706\") " pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 03:59:22.301538 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:22.301477 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-home\") pod \"premium-w5-56a44f-kserve-b7d69c94f-987lg\" (UID: \"6603cd37-3576-49ff-a87c-e307d4d35706\") " pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 03:59:22.402336 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:22.402298 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-model-cache\") pod \"premium-w5-56a44f-kserve-b7d69c94f-987lg\" (UID: \"6603cd37-3576-49ff-a87c-e307d4d35706\") " pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 03:59:22.402646 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:22.402615 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlqq7\" (UniqueName: \"kubernetes.io/projected/6603cd37-3576-49ff-a87c-e307d4d35706-kube-api-access-hlqq7\") pod \"premium-w5-56a44f-kserve-b7d69c94f-987lg\" (UID: \"6603cd37-3576-49ff-a87c-e307d4d35706\") " pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 03:59:22.402748 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:22.402702 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6603cd37-3576-49ff-a87c-e307d4d35706-tls-certs\") pod \"premium-w5-56a44f-kserve-b7d69c94f-987lg\" (UID: \"6603cd37-3576-49ff-a87c-e307d4d35706\") " pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 03:59:22.402748 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:22.402732 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-dshm\") pod \"premium-w5-56a44f-kserve-b7d69c94f-987lg\" (UID: \"6603cd37-3576-49ff-a87c-e307d4d35706\") " pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 03:59:22.402922 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:22.402784 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-home\") pod \"premium-w5-56a44f-kserve-b7d69c94f-987lg\" (UID: \"6603cd37-3576-49ff-a87c-e307d4d35706\") " pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 03:59:22.402922 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:22.402853 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-tmp-dir\") pod \"premium-w5-56a44f-kserve-b7d69c94f-987lg\" (UID: \"6603cd37-3576-49ff-a87c-e307d4d35706\") " pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 03:59:22.402922 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:22.402883 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-model-cache\") pod \"premium-w5-56a44f-kserve-b7d69c94f-987lg\" (UID: \"6603cd37-3576-49ff-a87c-e307d4d35706\") " pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 03:59:22.403573 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:22.403549 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-home\") pod \"premium-w5-56a44f-kserve-b7d69c94f-987lg\" (UID: \"6603cd37-3576-49ff-a87c-e307d4d35706\") " pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 03:59:22.403680 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:22.403574 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-tmp-dir\") pod \"premium-w5-56a44f-kserve-b7d69c94f-987lg\" (UID: \"6603cd37-3576-49ff-a87c-e307d4d35706\") " pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 03:59:22.405648 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:22.405627 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-dshm\") pod \"premium-w5-56a44f-kserve-b7d69c94f-987lg\" (UID: \"6603cd37-3576-49ff-a87c-e307d4d35706\") " pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 03:59:22.406237 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:22.406216 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6603cd37-3576-49ff-a87c-e307d4d35706-tls-certs\") pod \"premium-w5-56a44f-kserve-b7d69c94f-987lg\" (UID: \"6603cd37-3576-49ff-a87c-e307d4d35706\") " pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 03:59:22.416786 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:22.416725 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlqq7\" (UniqueName: \"kubernetes.io/projected/6603cd37-3576-49ff-a87c-e307d4d35706-kube-api-access-hlqq7\") pod \"premium-w5-56a44f-kserve-b7d69c94f-987lg\" (UID: \"6603cd37-3576-49ff-a87c-e307d4d35706\") " pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 03:59:22.585526 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:22.585481 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 03:59:22.874147 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:22.874117 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg"] Nov 04 03:59:22.876436 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:59:22.876391 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6603cd37_3576_49ff_a87c_e307d4d35706.slice/crio-59e0c7c0a746e65776b9f12edd3fd33fdb4f549c5bc8b602c6f6d7b70b984341 WatchSource:0}: Error finding container 59e0c7c0a746e65776b9f12edd3fd33fdb4f549c5bc8b602c6f6d7b70b984341: Status 404 returned error can't find the container with id 59e0c7c0a746e65776b9f12edd3fd33fdb4f549c5bc8b602c6f6d7b70b984341 Nov 04 03:59:23.375805 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:23.375734 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" event={"ID":"6603cd37-3576-49ff-a87c-e307d4d35706","Type":"ContainerStarted","Data":"179574c181fb521b043f92fb87e3834fd47ea2531a040f67e2f4c67d636e7abd"} Nov 04 03:59:23.376259 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:23.375821 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" event={"ID":"6603cd37-3576-49ff-a87c-e307d4d35706","Type":"ContainerStarted","Data":"59e0c7c0a746e65776b9f12edd3fd33fdb4f549c5bc8b602c6f6d7b70b984341"} Nov 04 03:59:23.401384 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:23.401328 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" podStartSLOduration=1.4013148530000001 podStartE2EDuration="1.401314853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:59:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:59:23.399712038 +0000 UTC m=+561.438152004" watchObservedRunningTime="2026-11-04 03:59:23.401314853 +0000 UTC m=+561.439754816" Nov 04 03:59:23.585951 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:23.585918 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 03:59:24.640831 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:24.640797 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 03:59:24.653817 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:24.653789 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 03:59:25.398433 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:25.398402 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 03:59:28.612572 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:28.612528 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-zf6m4"] Nov 04 03:59:32.586303 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:32.586262 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 03:59:32.598932 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:32.598901 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 03:59:33.447286 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:33.447248 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 03:59:35.838415 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:35.838372 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b"] Nov 04 03:59:35.843519 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:35.843489 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 03:59:35.852444 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:35.852372 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-4afe0e\"/\"premium-w0-4afe0e-kserve-self-signed-certs\"" Nov 04 03:59:35.854586 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:35.854541 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b"] Nov 04 03:59:35.953585 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:35.953540 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-tmp-dir\") pod \"premium-w0-4afe0e-kserve-6c5c95b95c-njj8b\" (UID: \"eef5b758-9117-4d9a-901c-29f03be11804\") " pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 03:59:35.953788 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:35.953626 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg4n2\" (UniqueName: \"kubernetes.io/projected/eef5b758-9117-4d9a-901c-29f03be11804-kube-api-access-vg4n2\") pod \"premium-w0-4afe0e-kserve-6c5c95b95c-njj8b\" (UID: \"eef5b758-9117-4d9a-901c-29f03be11804\") " pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 03:59:35.953788 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:35.953728 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-model-cache\") pod \"premium-w0-4afe0e-kserve-6c5c95b95c-njj8b\" (UID: \"eef5b758-9117-4d9a-901c-29f03be11804\") " pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 03:59:35.953889 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:35.953793 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eef5b758-9117-4d9a-901c-29f03be11804-tls-certs\") pod \"premium-w0-4afe0e-kserve-6c5c95b95c-njj8b\" (UID: \"eef5b758-9117-4d9a-901c-29f03be11804\") " pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 03:59:35.953889 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:35.953859 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-home\") pod \"premium-w0-4afe0e-kserve-6c5c95b95c-njj8b\" (UID: \"eef5b758-9117-4d9a-901c-29f03be11804\") " pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 03:59:35.953958 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:35.953887 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-dshm\") pod \"premium-w0-4afe0e-kserve-6c5c95b95c-njj8b\" (UID: \"eef5b758-9117-4d9a-901c-29f03be11804\") " pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 03:59:36.054776 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:36.054721 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-home\") pod \"premium-w0-4afe0e-kserve-6c5c95b95c-njj8b\" (UID: \"eef5b758-9117-4d9a-901c-29f03be11804\") " pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 03:59:36.054989 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:36.054796 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-dshm\") pod \"premium-w0-4afe0e-kserve-6c5c95b95c-njj8b\" (UID: \"eef5b758-9117-4d9a-901c-29f03be11804\") " pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 03:59:36.054989 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:36.054864 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-tmp-dir\") pod \"premium-w0-4afe0e-kserve-6c5c95b95c-njj8b\" (UID: \"eef5b758-9117-4d9a-901c-29f03be11804\") " pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 03:59:36.054989 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:36.054919 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg4n2\" (UniqueName: \"kubernetes.io/projected/eef5b758-9117-4d9a-901c-29f03be11804-kube-api-access-vg4n2\") pod \"premium-w0-4afe0e-kserve-6c5c95b95c-njj8b\" (UID: \"eef5b758-9117-4d9a-901c-29f03be11804\") " pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 03:59:36.055159 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:36.055035 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-model-cache\") pod \"premium-w0-4afe0e-kserve-6c5c95b95c-njj8b\" (UID: \"eef5b758-9117-4d9a-901c-29f03be11804\") " pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 03:59:36.055159 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:36.055066 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eef5b758-9117-4d9a-901c-29f03be11804-tls-certs\") pod \"premium-w0-4afe0e-kserve-6c5c95b95c-njj8b\" (UID: \"eef5b758-9117-4d9a-901c-29f03be11804\") " pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 03:59:36.055159 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:36.055097 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-home\") pod \"premium-w0-4afe0e-kserve-6c5c95b95c-njj8b\" (UID: \"eef5b758-9117-4d9a-901c-29f03be11804\") " pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 03:59:36.055432 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:36.055409 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-model-cache\") pod \"premium-w0-4afe0e-kserve-6c5c95b95c-njj8b\" (UID: \"eef5b758-9117-4d9a-901c-29f03be11804\") " pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 03:59:36.055508 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:36.055460 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-tmp-dir\") pod \"premium-w0-4afe0e-kserve-6c5c95b95c-njj8b\" (UID: \"eef5b758-9117-4d9a-901c-29f03be11804\") " pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 03:59:36.058035 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:36.057996 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-dshm\") pod \"premium-w0-4afe0e-kserve-6c5c95b95c-njj8b\" (UID: \"eef5b758-9117-4d9a-901c-29f03be11804\") " pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 03:59:36.058774 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:36.058735 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eef5b758-9117-4d9a-901c-29f03be11804-tls-certs\") pod \"premium-w0-4afe0e-kserve-6c5c95b95c-njj8b\" (UID: \"eef5b758-9117-4d9a-901c-29f03be11804\") " pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 03:59:36.068571 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:36.068532 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg4n2\" (UniqueName: \"kubernetes.io/projected/eef5b758-9117-4d9a-901c-29f03be11804-kube-api-access-vg4n2\") pod \"premium-w0-4afe0e-kserve-6c5c95b95c-njj8b\" (UID: \"eef5b758-9117-4d9a-901c-29f03be11804\") " pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 03:59:36.161844 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:36.161725 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 03:59:36.446707 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:36.446517 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b"] Nov 04 03:59:36.466966 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:36.466934 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" event={"ID":"eef5b758-9117-4d9a-901c-29f03be11804","Type":"ContainerStarted","Data":"295b5ab5a53b2aea3223670ea16eb053ce737dd9c2ba4ce5c9db53acda274276"} Nov 04 03:59:37.477321 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:37.477269 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" event={"ID":"eef5b758-9117-4d9a-901c-29f03be11804","Type":"ContainerStarted","Data":"f3d40f3ccaff13b2fe489adbd24cde10adcba58f226771edacaae160b6802440"} Nov 04 03:59:37.499298 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:37.499237 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" podStartSLOduration=2.49922078 podStartE2EDuration="2.49922078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:59:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:59:37.49711437 +0000 UTC m=+575.535554326" watchObservedRunningTime="2026-11-04 03:59:37.49922078 +0000 UTC m=+575.537660743" Nov 04 03:59:38.162012 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:38.161977 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 03:59:39.745403 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:39.745362 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz"] Nov 04 03:59:39.750046 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:39.750013 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 03:59:39.761103 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:39.758396 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-78ca37\"/\"premium-w2-78ca37-kserve-self-signed-certs\"" Nov 04 03:59:39.761103 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:39.759650 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz"] Nov 04 03:59:39.899779 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:39.899279 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d06ff75-d1d7-440e-98fe-90fd6045618d-tls-certs\") pod \"premium-w2-78ca37-kserve-97c65cff5-9wfzz\" (UID: \"4d06ff75-d1d7-440e-98fe-90fd6045618d\") " pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 03:59:39.899779 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:39.899362 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-dshm\") pod \"premium-w2-78ca37-kserve-97c65cff5-9wfzz\" (UID: \"4d06ff75-d1d7-440e-98fe-90fd6045618d\") " pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 03:59:39.899779 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:39.899404 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-home\") pod \"premium-w2-78ca37-kserve-97c65cff5-9wfzz\" (UID: \"4d06ff75-d1d7-440e-98fe-90fd6045618d\") " pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 03:59:39.899779 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:39.899430 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsnbc\" (UniqueName: \"kubernetes.io/projected/4d06ff75-d1d7-440e-98fe-90fd6045618d-kube-api-access-fsnbc\") pod \"premium-w2-78ca37-kserve-97c65cff5-9wfzz\" (UID: \"4d06ff75-d1d7-440e-98fe-90fd6045618d\") " pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 03:59:39.899779 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:39.899535 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-model-cache\") pod \"premium-w2-78ca37-kserve-97c65cff5-9wfzz\" (UID: \"4d06ff75-d1d7-440e-98fe-90fd6045618d\") " pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 03:59:39.899779 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:39.899556 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-tmp-dir\") pod \"premium-w2-78ca37-kserve-97c65cff5-9wfzz\" (UID: \"4d06ff75-d1d7-440e-98fe-90fd6045618d\") " pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 03:59:40.000880 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:40.000772 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-model-cache\") pod \"premium-w2-78ca37-kserve-97c65cff5-9wfzz\" (UID: \"4d06ff75-d1d7-440e-98fe-90fd6045618d\") " pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 03:59:40.000880 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:40.000832 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-tmp-dir\") pod \"premium-w2-78ca37-kserve-97c65cff5-9wfzz\" (UID: \"4d06ff75-d1d7-440e-98fe-90fd6045618d\") " pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 03:59:40.001092 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:40.000924 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d06ff75-d1d7-440e-98fe-90fd6045618d-tls-certs\") pod \"premium-w2-78ca37-kserve-97c65cff5-9wfzz\" (UID: \"4d06ff75-d1d7-440e-98fe-90fd6045618d\") " pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 03:59:40.001092 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:40.000959 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-dshm\") pod \"premium-w2-78ca37-kserve-97c65cff5-9wfzz\" (UID: \"4d06ff75-d1d7-440e-98fe-90fd6045618d\") " pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 03:59:40.001092 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:40.000982 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-home\") pod \"premium-w2-78ca37-kserve-97c65cff5-9wfzz\" (UID: \"4d06ff75-d1d7-440e-98fe-90fd6045618d\") " pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 03:59:40.001092 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:40.000998 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsnbc\" (UniqueName: \"kubernetes.io/projected/4d06ff75-d1d7-440e-98fe-90fd6045618d-kube-api-access-fsnbc\") pod \"premium-w2-78ca37-kserve-97c65cff5-9wfzz\" (UID: \"4d06ff75-d1d7-440e-98fe-90fd6045618d\") " pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 03:59:40.001285 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:40.001245 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-model-cache\") pod \"premium-w2-78ca37-kserve-97c65cff5-9wfzz\" (UID: \"4d06ff75-d1d7-440e-98fe-90fd6045618d\") " pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 03:59:40.001529 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:40.001508 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-tmp-dir\") pod \"premium-w2-78ca37-kserve-97c65cff5-9wfzz\" (UID: \"4d06ff75-d1d7-440e-98fe-90fd6045618d\") " pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 03:59:40.001782 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:40.001744 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-home\") pod \"premium-w2-78ca37-kserve-97c65cff5-9wfzz\" (UID: \"4d06ff75-d1d7-440e-98fe-90fd6045618d\") " pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 03:59:40.003853 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:40.003820 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-dshm\") pod \"premium-w2-78ca37-kserve-97c65cff5-9wfzz\" (UID: \"4d06ff75-d1d7-440e-98fe-90fd6045618d\") " pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 03:59:40.004162 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:40.004142 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d06ff75-d1d7-440e-98fe-90fd6045618d-tls-certs\") pod \"premium-w2-78ca37-kserve-97c65cff5-9wfzz\" (UID: \"4d06ff75-d1d7-440e-98fe-90fd6045618d\") " pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 03:59:40.015383 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:40.015347 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsnbc\" (UniqueName: \"kubernetes.io/projected/4d06ff75-d1d7-440e-98fe-90fd6045618d-kube-api-access-fsnbc\") pod \"premium-w2-78ca37-kserve-97c65cff5-9wfzz\" (UID: \"4d06ff75-d1d7-440e-98fe-90fd6045618d\") " pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 03:59:40.078807 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:40.078737 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 03:59:40.371576 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:40.369613 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz"] Nov 04 03:59:40.375413 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:59:40.375379 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d06ff75_d1d7_440e_98fe_90fd6045618d.slice/crio-b1ef72aa811cbf5c34271d2a3c0e4a156b1b58b2943b8c01bd4a5af6030c5907 WatchSource:0}: Error finding container b1ef72aa811cbf5c34271d2a3c0e4a156b1b58b2943b8c01bd4a5af6030c5907: Status 404 returned error can't find the container with id b1ef72aa811cbf5c34271d2a3c0e4a156b1b58b2943b8c01bd4a5af6030c5907 Nov 04 03:59:40.499693 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:40.499658 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" event={"ID":"4d06ff75-d1d7-440e-98fe-90fd6045618d","Type":"ContainerStarted","Data":"088f28147ea57978d96c7cf6c782acaeb0600835ba24d9c072418d651b4424f5"} Nov 04 03:59:40.499693 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:40.499700 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" event={"ID":"4d06ff75-d1d7-440e-98fe-90fd6045618d","Type":"ContainerStarted","Data":"b1ef72aa811cbf5c34271d2a3c0e4a156b1b58b2943b8c01bd4a5af6030c5907"} Nov 04 03:59:40.521552 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:40.521497 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" podStartSLOduration=1.521483478 podStartE2EDuration="1.521483478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:59:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:59:40.519341313 +0000 UTC m=+578.557781294" watchObservedRunningTime="2026-11-04 03:59:40.521483478 +0000 UTC m=+578.559923441" Nov 04 03:59:41.079625 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:41.079586 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 03:59:44.234420 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:44.234379 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-zf6m4"] Nov 04 03:59:46.162384 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:46.162339 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 03:59:46.176291 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:46.176250 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 03:59:46.553011 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:46.552977 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 03:59:50.082411 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:50.082377 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 03:59:50.095174 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:50.095136 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 03:59:50.586972 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:50.586904 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 03:59:50.751933 ip-10-0-129-134 kubenswrapper[6000]: E1104 03:59:50.751893 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 03:59:50.871490 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:50.871397 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg"] Nov 04 03:59:50.875380 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:50.875356 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:50.882465 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:50.882433 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-36730f-openshift-default-dockercfg-czsxm\"" Nov 04 03:59:50.888972 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:50.888940 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg"] Nov 04 03:59:50.923644 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:50.923600 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-data\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:50.923847 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:50.923787 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-token\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:50.923907 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:50.923850 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istiod-ca-cert\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:50.923952 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:50.923902 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-envoy\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:50.924028 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:50.924002 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-workload-socket\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:50.924137 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:50.924110 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-credential-socket\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:50.924173 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:50.924154 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh4ts\" (UniqueName: \"kubernetes.io/projected/5869a8f0-b341-4616-b258-e5f6c9bab8f5-kube-api-access-nh4ts\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:50.924213 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:50.924182 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-workload-certs\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:50.924255 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:50.924241 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-podinfo\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:51.025826 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.025724 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-credential-socket\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:51.026013 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.025875 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh4ts\" (UniqueName: \"kubernetes.io/projected/5869a8f0-b341-4616-b258-e5f6c9bab8f5-kube-api-access-nh4ts\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:51.026013 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.025909 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-workload-certs\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:51.026013 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.025943 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-podinfo\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:51.026013 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.026009 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-data\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:51.026247 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.026117 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-token\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:51.026247 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.026166 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-credential-socket\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:51.026247 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.026177 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istiod-ca-cert\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:51.026247 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.026243 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-envoy\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:51.026531 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.026297 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-workload-socket\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:51.026584 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.026564 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-workload-certs\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:51.026647 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.026598 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-workload-socket\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:51.026955 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.026926 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istiod-ca-cert\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:51.027148 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.027122 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-data\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:51.029327 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.029292 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-envoy\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:51.032623 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.032584 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-podinfo\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:51.038922 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.038891 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh4ts\" (UniqueName: \"kubernetes.io/projected/5869a8f0-b341-4616-b258-e5f6c9bab8f5-kube-api-access-nh4ts\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:51.039313 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.039285 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-token\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:51.191186 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.191096 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:51.494214 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.494187 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg"] Nov 04 03:59:51.496446 ip-10-0-129-134 kubenswrapper[6000]: W1104 03:59:51.496407 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5869a8f0_b341_4616_b258_e5f6c9bab8f5.slice/crio-e1463ce35807dc65cb447caca803fef6e6a7490c9d9b9592edb7493765234310 WatchSource:0}: Error finding container e1463ce35807dc65cb447caca803fef6e6a7490c9d9b9592edb7493765234310: Status 404 returned error can't find the container with id e1463ce35807dc65cb447caca803fef6e6a7490c9d9b9592edb7493765234310 Nov 04 03:59:51.499175 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.499133 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:59:51.499290 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.499229 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:59:51.499290 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.499269 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 03:59:51.585096 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.585056 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" event={"ID":"5869a8f0-b341-4616-b258-e5f6c9bab8f5","Type":"ContainerStarted","Data":"94926733e75992f8411eae66cdac09b45801589fb0ec3d065e4ee1cef2d254e8"} Nov 04 03:59:51.585279 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.585126 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" event={"ID":"5869a8f0-b341-4616-b258-e5f6c9bab8f5","Type":"ContainerStarted","Data":"e1463ce35807dc65cb447caca803fef6e6a7490c9d9b9592edb7493765234310"} Nov 04 03:59:51.609700 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:51.609516 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" podStartSLOduration=1.609496402 podStartE2EDuration="1.609496402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:59:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:59:51.609124631 +0000 UTC m=+589.647564597" watchObservedRunningTime="2026-11-04 03:59:51.609496402 +0000 UTC m=+589.647936371" Nov 04 03:59:52.191982 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:52.191946 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:52.193936 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:52.193886 6000 patch_prober.go:29] interesting pod/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Nov 04 03:59:52.194080 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:52.193968 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" podUID="5869a8f0-b341-4616-b258-e5f6c9bab8f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Nov 04 03:59:53.191813 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:53.191776 6000 patch_prober.go:29] interesting pod/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Nov 04 03:59:53.191977 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:53.191831 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" podUID="5869a8f0-b341-4616-b258-e5f6c9bab8f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Nov 04 03:59:54.192421 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:54.192385 6000 patch_prober.go:29] interesting pod/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Nov 04 03:59:54.192852 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:54.192447 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" podUID="5869a8f0-b341-4616-b258-e5f6c9bab8f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Nov 04 03:59:55.191450 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:55.191416 6000 patch_prober.go:29] interesting pod/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Nov 04 03:59:55.191666 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:55.191469 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" podUID="5869a8f0-b341-4616-b258-e5f6c9bab8f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Nov 04 03:59:56.192417 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:56.192383 6000 patch_prober.go:29] interesting pod/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Nov 04 03:59:56.192833 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:56.192435 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" podUID="5869a8f0-b341-4616-b258-e5f6c9bab8f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Nov 04 03:59:57.191886 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:57.191848 6000 patch_prober.go:29] interesting pod/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Nov 04 03:59:57.192053 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:57.191904 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" podUID="5869a8f0-b341-4616-b258-e5f6c9bab8f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Nov 04 03:59:58.195972 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:58.195939 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:58.196367 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:58.196344 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 03:59:58.197018 ip-10-0-129-134 kubenswrapper[6000]: I1104 03:59:58.197000 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 04:00:00.166263 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.163740 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt"] Nov 04 04:00:00.172727 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.172698 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt" Nov 04 04:00:00.182590 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.182561 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2wnx7\"" Nov 04 04:00:00.184204 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.184178 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt"] Nov 04 04:00:00.194189 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.194159 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv"] Nov 04 04:00:00.198974 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.198945 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv" Nov 04 04:00:00.199539 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.198832 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8"] Nov 04 04:00:00.204541 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.204512 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8" Nov 04 04:00:00.205027 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.205003 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv"] Nov 04 04:00:00.258778 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.255690 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0b6cd3-c2b2-43af-94df-aab3c8d837f5-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt\" (UID: \"bf0b6cd3-c2b2-43af-94df-aab3c8d837f5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt" Nov 04 04:00:00.258778 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.255823 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1426426-d88c-474a-8e0c-b532d0f31756-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8\" (UID: \"d1426426-d88c-474a-8e0c-b532d0f31756\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8" Nov 04 04:00:00.258778 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.255853 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2cpt\" (UniqueName: \"kubernetes.io/projected/d1426426-d88c-474a-8e0c-b532d0f31756-kube-api-access-t2cpt\") pod \"maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8\" (UID: \"d1426426-d88c-474a-8e0c-b532d0f31756\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8" Nov 04 04:00:00.258778 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.255909 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dss57\" (UniqueName: \"kubernetes.io/projected/a6c5e197-376e-4303-aa47-55057206aa33-kube-api-access-dss57\") pod \"maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv\" (UID: \"a6c5e197-376e-4303-aa47-55057206aa33\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv" Nov 04 04:00:00.258778 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.255951 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvzxp\" (UniqueName: \"kubernetes.io/projected/bf0b6cd3-c2b2-43af-94df-aab3c8d837f5-kube-api-access-mvzxp\") pod \"maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt\" (UID: \"bf0b6cd3-c2b2-43af-94df-aab3c8d837f5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt" Nov 04 04:00:00.258778 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.256141 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6c5e197-376e-4303-aa47-55057206aa33-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv\" (UID: \"a6c5e197-376e-4303-aa47-55057206aa33\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv" Nov 04 04:00:00.283782 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.283337 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8"] Nov 04 04:00:00.357539 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.357504 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dss57\" (UniqueName: \"kubernetes.io/projected/a6c5e197-376e-4303-aa47-55057206aa33-kube-api-access-dss57\") pod \"maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv\" (UID: \"a6c5e197-376e-4303-aa47-55057206aa33\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv" Nov 04 04:00:00.357743 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.357552 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvzxp\" (UniqueName: \"kubernetes.io/projected/bf0b6cd3-c2b2-43af-94df-aab3c8d837f5-kube-api-access-mvzxp\") pod \"maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt\" (UID: \"bf0b6cd3-c2b2-43af-94df-aab3c8d837f5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt" Nov 04 04:00:00.357743 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.357620 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6c5e197-376e-4303-aa47-55057206aa33-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv\" (UID: \"a6c5e197-376e-4303-aa47-55057206aa33\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv" Nov 04 04:00:00.357937 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.357818 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0b6cd3-c2b2-43af-94df-aab3c8d837f5-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt\" (UID: \"bf0b6cd3-c2b2-43af-94df-aab3c8d837f5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt" Nov 04 04:00:00.358003 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.357963 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1426426-d88c-474a-8e0c-b532d0f31756-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8\" (UID: \"d1426426-d88c-474a-8e0c-b532d0f31756\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8" Nov 04 04:00:00.358003 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.357991 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2cpt\" (UniqueName: \"kubernetes.io/projected/d1426426-d88c-474a-8e0c-b532d0f31756-kube-api-access-t2cpt\") pod \"maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8\" (UID: \"d1426426-d88c-474a-8e0c-b532d0f31756\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8" Nov 04 04:00:00.358478 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.358451 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6c5e197-376e-4303-aa47-55057206aa33-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv\" (UID: \"a6c5e197-376e-4303-aa47-55057206aa33\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv" Nov 04 04:00:00.358677 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.358653 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0b6cd3-c2b2-43af-94df-aab3c8d837f5-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt\" (UID: \"bf0b6cd3-c2b2-43af-94df-aab3c8d837f5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt" Nov 04 04:00:00.358911 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.358893 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1426426-d88c-474a-8e0c-b532d0f31756-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8\" (UID: \"d1426426-d88c-474a-8e0c-b532d0f31756\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8" Nov 04 04:00:00.382775 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.382730 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dss57\" (UniqueName: \"kubernetes.io/projected/a6c5e197-376e-4303-aa47-55057206aa33-kube-api-access-dss57\") pod \"maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv\" (UID: \"a6c5e197-376e-4303-aa47-55057206aa33\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv" Nov 04 04:00:00.389662 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.389629 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvzxp\" (UniqueName: \"kubernetes.io/projected/bf0b6cd3-c2b2-43af-94df-aab3c8d837f5-kube-api-access-mvzxp\") pod \"maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt\" (UID: \"bf0b6cd3-c2b2-43af-94df-aab3c8d837f5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt" Nov 04 04:00:00.389951 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.389718 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2cpt\" (UniqueName: \"kubernetes.io/projected/d1426426-d88c-474a-8e0c-b532d0f31756-kube-api-access-t2cpt\") pod \"maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8\" (UID: \"d1426426-d88c-474a-8e0c-b532d0f31756\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8" Nov 04 04:00:00.501541 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.501447 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt" Nov 04 04:00:00.556333 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.555981 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv" Nov 04 04:00:00.564342 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.564305 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8" Nov 04 04:00:00.800717 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.800690 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt"] Nov 04 04:00:00.802598 ip-10-0-129-134 kubenswrapper[6000]: W1104 04:00:00.802563 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf0b6cd3_c2b2_43af_94df_aab3c8d837f5.slice/crio-d467225f53308d3f762cee5b64f50740d54a498db5264bf6573a4fb284e297a0 WatchSource:0}: Error finding container d467225f53308d3f762cee5b64f50740d54a498db5264bf6573a4fb284e297a0: Status 404 returned error can't find the container with id d467225f53308d3f762cee5b64f50740d54a498db5264bf6573a4fb284e297a0 Nov 04 04:00:00.886237 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.886205 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv"] Nov 04 04:00:00.888311 ip-10-0-129-134 kubenswrapper[6000]: W1104 04:00:00.888282 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6c5e197_376e_4303_aa47_55057206aa33.slice/crio-32dd71ecd2cfa6fc2d466f04c1c090ebc27272627c3f11b0c5d00ad898680480 WatchSource:0}: Error finding container 32dd71ecd2cfa6fc2d466f04c1c090ebc27272627c3f11b0c5d00ad898680480: Status 404 returned error can't find the container with id 32dd71ecd2cfa6fc2d466f04c1c090ebc27272627c3f11b0c5d00ad898680480 Nov 04 04:00:00.889646 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:00.889617 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8"] Nov 04 04:00:00.890767 ip-10-0-129-134 kubenswrapper[6000]: W1104 04:00:00.890730 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1426426_d88c_474a_8e0c_b532d0f31756.slice/crio-5a41453c98ae2be060d89fd0af97e0407121617f6bb1fab3c8fe3cc95f6a6085 WatchSource:0}: Error finding container 5a41453c98ae2be060d89fd0af97e0407121617f6bb1fab3c8fe3cc95f6a6085: Status 404 returned error can't find the container with id 5a41453c98ae2be060d89fd0af97e0407121617f6bb1fab3c8fe3cc95f6a6085 Nov 04 04:00:01.708833 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:01.708789 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8" event={"ID":"d1426426-d88c-474a-8e0c-b532d0f31756","Type":"ContainerStarted","Data":"5a41453c98ae2be060d89fd0af97e0407121617f6bb1fab3c8fe3cc95f6a6085"} Nov 04 04:00:01.710258 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:01.710224 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv" event={"ID":"a6c5e197-376e-4303-aa47-55057206aa33","Type":"ContainerStarted","Data":"32dd71ecd2cfa6fc2d466f04c1c090ebc27272627c3f11b0c5d00ad898680480"} Nov 04 04:00:01.711589 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:01.711561 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt" event={"ID":"bf0b6cd3-c2b2-43af-94df-aab3c8d837f5","Type":"ContainerStarted","Data":"d467225f53308d3f762cee5b64f50740d54a498db5264bf6573a4fb284e297a0"} Nov 04 04:00:02.163454 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:02.163415 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zfgzh_42510c3d-0283-4901-9d23-a84793598ac9/ovn-acl-logging/0.log" Nov 04 04:00:02.164028 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:02.164009 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zfgzh_42510c3d-0283-4901-9d23-a84793598ac9/ovn-acl-logging/0.log" Nov 04 04:00:05.340616 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.340580 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc"] Nov 04 04:00:05.344504 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.344480 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.351080 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.351053 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-998997-openshift-default-dockercfg-sksxz\"" Nov 04 04:00:05.357894 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.357861 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc"] Nov 04 04:00:05.416943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.416912 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-token\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.417098 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.416985 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq6nt\" (UniqueName: \"kubernetes.io/projected/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-kube-api-access-rq6nt\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.417098 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.417046 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-envoy\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.417172 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.417146 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istiod-ca-cert\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.417271 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.417189 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-workload-certs\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.417271 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.417204 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-data\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.417271 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.417243 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-workload-socket\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.417426 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.417346 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-podinfo\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.417474 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.417435 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-credential-socket\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.518490 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.518418 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-podinfo\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.518490 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.518487 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-credential-socket\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.518673 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.518525 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-token\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.518673 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.518583 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq6nt\" (UniqueName: \"kubernetes.io/projected/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-kube-api-access-rq6nt\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.518673 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.518637 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-envoy\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.518830 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.518700 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istiod-ca-cert\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.518830 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.518745 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-workload-certs\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.518830 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.518795 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-data\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.518830 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.518820 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-workload-socket\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.519076 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.518908 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-credential-socket\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.519164 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.519127 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-workload-certs\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.519224 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.519171 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-workload-socket\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.519280 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.519242 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-data\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.519507 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.519487 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istiod-ca-cert\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.520904 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.520879 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-envoy\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.521000 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.520972 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-podinfo\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.610177 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.609906 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-token\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.610177 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.610117 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq6nt\" (UniqueName: \"kubernetes.io/projected/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-kube-api-access-rq6nt\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-6f2pc\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.675582 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.675547 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:05.780455 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.780361 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8" event={"ID":"d1426426-d88c-474a-8e0c-b532d0f31756","Type":"ContainerStarted","Data":"d3bcf8f40e7ea6189f8f0a168ad83a967657008a80377121b9c6f97addaf6990"} Nov 04 04:00:05.783483 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.783148 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv" event={"ID":"a6c5e197-376e-4303-aa47-55057206aa33","Type":"ContainerStarted","Data":"a0f4419a272a5b1d627bbd8cf34fd8020c4d4513e9647ed80773a5df44356d59"} Nov 04 04:00:05.785443 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.785407 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt" event={"ID":"bf0b6cd3-c2b2-43af-94df-aab3c8d837f5","Type":"ContainerStarted","Data":"6c920dbf58cf3a1cf85611adcf251752e815fcbfb3b23536f68ae803bf839761"} Nov 04 04:00:05.825897 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.825838 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt" podStartSLOduration=1.6491357789999999 podStartE2EDuration="5.825811686s" totalImagesPullingTime="4.176675907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:00:05.819944258 +0000 UTC m=+603.858384239" watchObservedRunningTime="2026-11-04 04:00:05.825811686 +0000 UTC m=+603.864251655" Nov 04 04:00:05.826736 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.826688 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8" podStartSLOduration=1.47689401 podStartE2EDuration="5.826676541s" totalImagesPullingTime="4.349782531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:00:05.802367716 +0000 UTC m=+603.840807683" watchObservedRunningTime="2026-11-04 04:00:05.826676541 +0000 UTC m=+603.865116506" Nov 04 04:00:05.840458 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.840402 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv" podStartSLOduration=1.49868445 podStartE2EDuration="5.840385357s" totalImagesPullingTime="4.341700907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:00:05.838831928 +0000 UTC m=+603.877271895" watchObservedRunningTime="2026-11-04 04:00:05.840385357 +0000 UTC m=+603.878825321" Nov 04 04:00:05.982832 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.982789 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc"] Nov 04 04:00:05.984347 ip-10-0-129-134 kubenswrapper[6000]: W1104 04:00:05.984317 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48bb22c3_c4b7_45e3_ace0_b7b33b94fa2d.slice/crio-0dad8552b1d01b128096674e5ceff65b89624a9867dfba7ca3ee30edd579df2d WatchSource:0}: Error finding container 0dad8552b1d01b128096674e5ceff65b89624a9867dfba7ca3ee30edd579df2d: Status 404 returned error can't find the container with id 0dad8552b1d01b128096674e5ceff65b89624a9867dfba7ca3ee30edd579df2d Nov 04 04:00:05.987072 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.987039 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:00:05.987162 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.987106 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:00:05.987162 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:05.987136 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:00:06.794244 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:06.794202 6000 generic.go:356] "Generic (PLEG): container finished" podID="bf0b6cd3-c2b2-43af-94df-aab3c8d837f5" containerID="6c920dbf58cf3a1cf85611adcf251752e815fcbfb3b23536f68ae803bf839761" exitCode=60 Nov 04 04:00:06.794715 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:06.794286 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt" event={"ID":"bf0b6cd3-c2b2-43af-94df-aab3c8d837f5","Type":"ContainerDied","Data":"6c920dbf58cf3a1cf85611adcf251752e815fcbfb3b23536f68ae803bf839761"} Nov 04 04:00:06.794715 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:06.794589 6000 scope.go:122] "RemoveContainer" containerID="6c920dbf58cf3a1cf85611adcf251752e815fcbfb3b23536f68ae803bf839761" Nov 04 04:00:06.796285 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:06.796260 6000 generic.go:356] "Generic (PLEG): container finished" podID="d1426426-d88c-474a-8e0c-b532d0f31756" containerID="d3bcf8f40e7ea6189f8f0a168ad83a967657008a80377121b9c6f97addaf6990" exitCode=60 Nov 04 04:00:06.796402 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:06.796385 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8" event={"ID":"d1426426-d88c-474a-8e0c-b532d0f31756","Type":"ContainerDied","Data":"d3bcf8f40e7ea6189f8f0a168ad83a967657008a80377121b9c6f97addaf6990"} Nov 04 04:00:06.796727 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:06.796712 6000 scope.go:122] "RemoveContainer" containerID="d3bcf8f40e7ea6189f8f0a168ad83a967657008a80377121b9c6f97addaf6990" Nov 04 04:00:06.799508 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:06.799406 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" event={"ID":"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d","Type":"ContainerStarted","Data":"c7d765357c5924ff5a248cb7e9f029accb6d15fc803813ec2214e341b8a636d8"} Nov 04 04:00:06.799508 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:06.799455 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" event={"ID":"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d","Type":"ContainerStarted","Data":"0dad8552b1d01b128096674e5ceff65b89624a9867dfba7ca3ee30edd579df2d"} Nov 04 04:00:06.801473 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:06.801451 6000 generic.go:356] "Generic (PLEG): container finished" podID="a6c5e197-376e-4303-aa47-55057206aa33" containerID="a0f4419a272a5b1d627bbd8cf34fd8020c4d4513e9647ed80773a5df44356d59" exitCode=60 Nov 04 04:00:06.801578 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:06.801493 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv" event={"ID":"a6c5e197-376e-4303-aa47-55057206aa33","Type":"ContainerDied","Data":"a0f4419a272a5b1d627bbd8cf34fd8020c4d4513e9647ed80773a5df44356d59"} Nov 04 04:00:06.801849 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:06.801818 6000 scope.go:122] "RemoveContainer" containerID="a0f4419a272a5b1d627bbd8cf34fd8020c4d4513e9647ed80773a5df44356d59" Nov 04 04:00:06.865439 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:06.865340 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" podStartSLOduration=1.8653194640000001 podStartE2EDuration="1.865319464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:00:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:00:06.859672756 +0000 UTC m=+604.898112751" watchObservedRunningTime="2026-11-04 04:00:06.865319464 +0000 UTC m=+604.903759431" Nov 04 04:00:07.676932 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:07.676687 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:07.678808 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:07.678773 6000 patch_prober.go:29] interesting pod/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Nov 04 04:00:07.678965 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:07.678832 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" podUID="48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Nov 04 04:00:07.819792 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:07.819379 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv" event={"ID":"a6c5e197-376e-4303-aa47-55057206aa33","Type":"ContainerStarted","Data":"f347bf5344a8bbb8112768ec90b0c29c8e08139a48d927f66b9d97cb3c6dfe26"} Nov 04 04:00:07.844521 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:07.844484 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt" event={"ID":"bf0b6cd3-c2b2-43af-94df-aab3c8d837f5","Type":"ContainerStarted","Data":"8e60b1c28298304056a2e45a23a4e119c9b1e5edeb7774a1f3f5b9e8f03f836d"} Nov 04 04:00:07.846926 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:07.846900 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8" event={"ID":"d1426426-d88c-474a-8e0c-b532d0f31756","Type":"ContainerStarted","Data":"f8e0a101d670f8fd33e4478e6ea578480294216a39112b8ce07ec03b81168979"} Nov 04 04:00:08.676855 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:08.676816 6000 patch_prober.go:29] interesting pod/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Nov 04 04:00:08.677086 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:08.676870 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" podUID="48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Nov 04 04:00:08.862678 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:08.862643 6000 generic.go:356] "Generic (PLEG): container finished" podID="d1426426-d88c-474a-8e0c-b532d0f31756" containerID="f8e0a101d670f8fd33e4478e6ea578480294216a39112b8ce07ec03b81168979" exitCode=60 Nov 04 04:00:08.863091 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:08.862788 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8" event={"ID":"d1426426-d88c-474a-8e0c-b532d0f31756","Type":"ContainerDied","Data":"f8e0a101d670f8fd33e4478e6ea578480294216a39112b8ce07ec03b81168979"} Nov 04 04:00:08.863091 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:08.862838 6000 scope.go:122] "RemoveContainer" containerID="d3bcf8f40e7ea6189f8f0a168ad83a967657008a80377121b9c6f97addaf6990" Nov 04 04:00:08.863209 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:08.863194 6000 scope.go:122] "RemoveContainer" containerID="f8e0a101d670f8fd33e4478e6ea578480294216a39112b8ce07ec03b81168979" Nov 04 04:00:08.863454 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:00:08.863437 6000 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8_odh-ai-gateway-infra(d1426426-d88c-474a-8e0c-b532d0f31756)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8" podUID="d1426426-d88c-474a-8e0c-b532d0f31756" Nov 04 04:00:08.866105 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:08.866076 6000 generic.go:356] "Generic (PLEG): container finished" podID="a6c5e197-376e-4303-aa47-55057206aa33" containerID="f347bf5344a8bbb8112768ec90b0c29c8e08139a48d927f66b9d97cb3c6dfe26" exitCode=60 Nov 04 04:00:08.866242 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:08.866177 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv" event={"ID":"a6c5e197-376e-4303-aa47-55057206aa33","Type":"ContainerDied","Data":"f347bf5344a8bbb8112768ec90b0c29c8e08139a48d927f66b9d97cb3c6dfe26"} Nov 04 04:00:08.866411 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:08.866393 6000 scope.go:122] "RemoveContainer" containerID="f347bf5344a8bbb8112768ec90b0c29c8e08139a48d927f66b9d97cb3c6dfe26" Nov 04 04:00:08.866682 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:00:08.866663 6000 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv_odh-ai-gateway-infra(a6c5e197-376e-4303-aa47-55057206aa33)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv" podUID="a6c5e197-376e-4303-aa47-55057206aa33" Nov 04 04:00:08.868378 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:08.868343 6000 generic.go:356] "Generic (PLEG): container finished" podID="bf0b6cd3-c2b2-43af-94df-aab3c8d837f5" containerID="8e60b1c28298304056a2e45a23a4e119c9b1e5edeb7774a1f3f5b9e8f03f836d" exitCode=60 Nov 04 04:00:08.869367 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:08.869345 6000 scope.go:122] "RemoveContainer" containerID="8e60b1c28298304056a2e45a23a4e119c9b1e5edeb7774a1f3f5b9e8f03f836d" Nov 04 04:00:08.869589 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:00:08.869567 6000 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt_odh-ai-gateway-infra(bf0b6cd3-c2b2-43af-94df-aab3c8d837f5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt" podUID="bf0b6cd3-c2b2-43af-94df-aab3c8d837f5" Nov 04 04:00:08.869641 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:08.869616 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt" event={"ID":"bf0b6cd3-c2b2-43af-94df-aab3c8d837f5","Type":"ContainerDied","Data":"8e60b1c28298304056a2e45a23a4e119c9b1e5edeb7774a1f3f5b9e8f03f836d"} Nov 04 04:00:08.877889 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:08.877837 6000 scope.go:122] "RemoveContainer" containerID="a0f4419a272a5b1d627bbd8cf34fd8020c4d4513e9647ed80773a5df44356d59" Nov 04 04:00:08.890879 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:08.890859 6000 scope.go:122] "RemoveContainer" containerID="6c920dbf58cf3a1cf85611adcf251752e815fcbfb3b23536f68ae803bf839761" Nov 04 04:00:09.676051 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:09.676009 6000 patch_prober.go:29] interesting pod/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Nov 04 04:00:09.676258 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:09.676078 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" podUID="48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Nov 04 04:00:09.878314 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:09.878286 6000 scope.go:122] "RemoveContainer" containerID="f8e0a101d670f8fd33e4478e6ea578480294216a39112b8ce07ec03b81168979" Nov 04 04:00:09.878736 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:00:09.878549 6000 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8_odh-ai-gateway-infra(d1426426-d88c-474a-8e0c-b532d0f31756)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8" podUID="d1426426-d88c-474a-8e0c-b532d0f31756" Nov 04 04:00:09.879896 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:09.879869 6000 scope.go:122] "RemoveContainer" containerID="f347bf5344a8bbb8112768ec90b0c29c8e08139a48d927f66b9d97cb3c6dfe26" Nov 04 04:00:09.880142 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:00:09.880117 6000 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv_odh-ai-gateway-infra(a6c5e197-376e-4303-aa47-55057206aa33)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv" podUID="a6c5e197-376e-4303-aa47-55057206aa33" Nov 04 04:00:09.881637 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:09.881618 6000 scope.go:122] "RemoveContainer" containerID="8e60b1c28298304056a2e45a23a4e119c9b1e5edeb7774a1f3f5b9e8f03f836d" Nov 04 04:00:09.881857 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:00:09.881827 6000 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt_odh-ai-gateway-infra(bf0b6cd3-c2b2-43af-94df-aab3c8d837f5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt" podUID="bf0b6cd3-c2b2-43af-94df-aab3c8d837f5" Nov 04 04:00:10.676391 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:10.676351 6000 patch_prober.go:29] interesting pod/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Nov 04 04:00:10.676558 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:10.676407 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" podUID="48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Nov 04 04:00:11.676396 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:11.676357 6000 patch_prober.go:29] interesting pod/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Nov 04 04:00:11.676940 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:11.676421 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" podUID="48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Nov 04 04:00:12.043589 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:12.043558 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-zf6m4"] Nov 04 04:00:13.683612 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:13.683564 6000 patch_prober.go:29] interesting pod/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 04:00:13.684045 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:13.683642 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" podUID="48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 04:00:13.725665 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:13.725608 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:13.726533 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:13.726503 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:13.727338 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:13.727314 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:00:18.082537 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:18.082500 6000 scope.go:122] "RemoveContainer" containerID="f8e0a101d670f8fd33e4478e6ea578480294216a39112b8ce07ec03b81168979" Nov 04 04:00:18.959654 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:18.959615 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8" event={"ID":"d1426426-d88c-474a-8e0c-b532d0f31756","Type":"ContainerStarted","Data":"53c6424cabd1963e944558ba5cd7c4dcc396f56c2765cc2ca881764b2ec64ab8"} Nov 04 04:00:19.968390 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:19.968352 6000 generic.go:356] "Generic (PLEG): container finished" podID="d1426426-d88c-474a-8e0c-b532d0f31756" containerID="53c6424cabd1963e944558ba5cd7c4dcc396f56c2765cc2ca881764b2ec64ab8" exitCode=60 Nov 04 04:00:19.968878 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:19.968414 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8" event={"ID":"d1426426-d88c-474a-8e0c-b532d0f31756","Type":"ContainerDied","Data":"53c6424cabd1963e944558ba5cd7c4dcc396f56c2765cc2ca881764b2ec64ab8"} Nov 04 04:00:19.968878 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:19.968476 6000 scope.go:122] "RemoveContainer" containerID="f8e0a101d670f8fd33e4478e6ea578480294216a39112b8ce07ec03b81168979" Nov 04 04:00:19.968878 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:19.968837 6000 scope.go:122] "RemoveContainer" containerID="53c6424cabd1963e944558ba5cd7c4dcc396f56c2765cc2ca881764b2ec64ab8" Nov 04 04:00:19.969133 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:00:19.969111 6000 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8_odh-ai-gateway-infra(d1426426-d88c-474a-8e0c-b532d0f31756)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8" podUID="d1426426-d88c-474a-8e0c-b532d0f31756" Nov 04 04:00:19.994903 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:19.994869 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8"] Nov 04 04:00:20.749357 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:00:20.749289 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 04:00:21.075423 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.075389 6000 scope.go:122] "RemoveContainer" containerID="8e60b1c28298304056a2e45a23a4e119c9b1e5edeb7774a1f3f5b9e8f03f836d" Nov 04 04:00:21.076289 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.076266 6000 scope.go:122] "RemoveContainer" containerID="f347bf5344a8bbb8112768ec90b0c29c8e08139a48d927f66b9d97cb3c6dfe26" Nov 04 04:00:21.151625 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.151597 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8" Nov 04 04:00:21.203806 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.203772 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w"] Nov 04 04:00:21.204262 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.204249 6000 cpu_manager.go:414] "containerMap: removing container" podUID="d1426426-d88c-474a-8e0c-b532d0f31756" containerName="cleanup" Nov 04 04:00:21.204329 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.204263 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1426426-d88c-474a-8e0c-b532d0f31756" containerName="cleanup" Nov 04 04:00:21.204329 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.204276 6000 cpu_manager.go:414] "containerMap: removing container" podUID="d1426426-d88c-474a-8e0c-b532d0f31756" containerName="cleanup" Nov 04 04:00:21.204329 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.204282 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1426426-d88c-474a-8e0c-b532d0f31756" containerName="cleanup" Nov 04 04:00:21.204329 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.204290 6000 cpu_manager.go:414] "containerMap: removing container" podUID="d1426426-d88c-474a-8e0c-b532d0f31756" containerName="cleanup" Nov 04 04:00:21.204329 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.204295 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1426426-d88c-474a-8e0c-b532d0f31756" containerName="cleanup" Nov 04 04:00:21.204499 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.204369 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1426426-d88c-474a-8e0c-b532d0f31756" containerName="cleanup" Nov 04 04:00:21.204499 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.204377 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1426426-d88c-474a-8e0c-b532d0f31756" containerName="cleanup" Nov 04 04:00:21.204499 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.204383 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1426426-d88c-474a-8e0c-b532d0f31756" containerName="cleanup" Nov 04 04:00:21.207707 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.207685 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.225806 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.214919 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-78ca37-openshift-default-dockercfg-84ldj\"" Nov 04 04:00:21.234160 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.230926 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1426426-d88c-474a-8e0c-b532d0f31756-kube-api-access-t2cpt\" (UniqueName: \"kubernetes.io/projected/d1426426-d88c-474a-8e0c-b532d0f31756-kube-api-access-t2cpt\") pod \"d1426426-d88c-474a-8e0c-b532d0f31756\" (UID: \"d1426426-d88c-474a-8e0c-b532d0f31756\") " Nov 04 04:00:21.234160 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.231015 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d1426426-d88c-474a-8e0c-b532d0f31756-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1426426-d88c-474a-8e0c-b532d0f31756-maas-api-service-ca\") pod \"d1426426-d88c-474a-8e0c-b532d0f31756\" (UID: \"d1426426-d88c-474a-8e0c-b532d0f31756\") " Nov 04 04:00:21.234160 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.232932 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1426426-d88c-474a-8e0c-b532d0f31756-maas-api-service-ca" pod "d1426426-d88c-474a-8e0c-b532d0f31756" (UID: "d1426426-d88c-474a-8e0c-b532d0f31756"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:00:21.234160 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.233800 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w"] Nov 04 04:00:21.310236 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.310198 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1426426-d88c-474a-8e0c-b532d0f31756-kube-api-access-t2cpt" pod "d1426426-d88c-474a-8e0c-b532d0f31756" (UID: "d1426426-d88c-474a-8e0c-b532d0f31756"). InnerVolumeSpecName "kube-api-access-t2cpt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:00:21.333332 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.333289 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/599a125f-8022-4f5a-9384-e54dd8e13b32-istiod-ca-cert\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.333527 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.333346 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-workload-socket\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.333527 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.333411 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-data\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.333652 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.333543 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-workload-certs\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.333652 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.333644 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-envoy\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.333740 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.333670 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-podinfo\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.333740 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.333691 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-token\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.333869 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.333743 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5x69\" (UniqueName: \"kubernetes.io/projected/599a125f-8022-4f5a-9384-e54dd8e13b32-kube-api-access-m5x69\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.333869 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.333859 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-credential-socket\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.333979 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.333966 6000 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1426426-d88c-474a-8e0c-b532d0f31756-maas-api-service-ca\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:00:21.334013 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.333984 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2cpt\" (UniqueName: \"kubernetes.io/projected/d1426426-d88c-474a-8e0c-b532d0f31756-kube-api-access-t2cpt\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:00:21.434917 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.434879 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-credential-socket\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.435071 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.434987 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/599a125f-8022-4f5a-9384-e54dd8e13b32-istiod-ca-cert\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.435071 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.435029 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-workload-socket\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.435071 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.435057 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-data\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.435247 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.435117 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-workload-certs\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.435247 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.435185 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-envoy\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.435247 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.435217 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-podinfo\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.435403 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.435248 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-token\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.435403 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.435284 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5x69\" (UniqueName: \"kubernetes.io/projected/599a125f-8022-4f5a-9384-e54dd8e13b32-kube-api-access-m5x69\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.435403 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.435331 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-credential-socket\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.435403 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.435341 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-workload-socket\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.435604 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.435492 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-data\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.435847 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.435808 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/599a125f-8022-4f5a-9384-e54dd8e13b32-istiod-ca-cert\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.436083 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.436056 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-workload-certs\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.443850 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.437722 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-envoy\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.443850 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.438281 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-podinfo\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.448571 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.448267 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-token\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.450278 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.448728 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5x69\" (UniqueName: \"kubernetes.io/projected/599a125f-8022-4f5a-9384-e54dd8e13b32-kube-api-access-m5x69\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.542089 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.542055 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:21.831879 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.831851 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w"] Nov 04 04:00:21.833745 ip-10-0-129-134 kubenswrapper[6000]: W1104 04:00:21.833717 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod599a125f_8022_4f5a_9384_e54dd8e13b32.slice/crio-526addb02caab5f265c1ad90713ce1f1dde78d4e1e2930181406c6fe5255f1b8 WatchSource:0}: Error finding container 526addb02caab5f265c1ad90713ce1f1dde78d4e1e2930181406c6fe5255f1b8: Status 404 returned error can't find the container with id 526addb02caab5f265c1ad90713ce1f1dde78d4e1e2930181406c6fe5255f1b8 Nov 04 04:00:21.836559 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.836526 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:00:21.836625 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.836596 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:00:21.836657 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.836622 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:00:21.984777 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.984118 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8" Nov 04 04:00:21.984777 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.984113 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8" event={"ID":"d1426426-d88c-474a-8e0c-b532d0f31756","Type":"ContainerDied","Data":"5a41453c98ae2be060d89fd0af97e0407121617f6bb1fab3c8fe3cc95f6a6085"} Nov 04 04:00:21.984777 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.984224 6000 scope.go:122] "RemoveContainer" containerID="53c6424cabd1963e944558ba5cd7c4dcc396f56c2765cc2ca881764b2ec64ab8" Nov 04 04:00:21.989193 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.986683 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv" event={"ID":"a6c5e197-376e-4303-aa47-55057206aa33","Type":"ContainerStarted","Data":"a716965fbc9d79f3911e58c9a567cd716cc3d900b81474afa5edf9e92b01aace"} Nov 04 04:00:21.991792 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.990195 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt" event={"ID":"bf0b6cd3-c2b2-43af-94df-aab3c8d837f5","Type":"ContainerStarted","Data":"03f2d364619d87f3e3b10fb82fb2adb320a11450111998880406f53c962c693b"} Nov 04 04:00:21.993532 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.993503 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" event={"ID":"599a125f-8022-4f5a-9384-e54dd8e13b32","Type":"ContainerStarted","Data":"75507b9736352520d19b589257ee1dd35be205c5986d354d8df2188362afd4b2"} Nov 04 04:00:21.993672 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:21.993558 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" event={"ID":"599a125f-8022-4f5a-9384-e54dd8e13b32","Type":"ContainerStarted","Data":"526addb02caab5f265c1ad90713ce1f1dde78d4e1e2930181406c6fe5255f1b8"} Nov 04 04:00:22.041300 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:22.040554 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" podStartSLOduration=1.040537362 podStartE2EDuration="1.040537362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:00:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:00:22.040317254 +0000 UTC m=+620.078757223" watchObservedRunningTime="2026-11-04 04:00:22.040537362 +0000 UTC m=+620.078977326" Nov 04 04:00:22.071966 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:22.071927 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8"] Nov 04 04:00:22.082708 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:22.082676 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-64c20d-29896080-d5gt8"] Nov 04 04:00:22.543841 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:22.543137 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:22.547791 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:22.547742 6000 patch_prober.go:29] interesting pod/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Nov 04 04:00:22.547962 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:22.547817 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" podUID="599a125f-8022-4f5a-9384-e54dd8e13b32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Nov 04 04:00:23.001664 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:23.001572 6000 generic.go:356] "Generic (PLEG): container finished" podID="a6c5e197-376e-4303-aa47-55057206aa33" containerID="a716965fbc9d79f3911e58c9a567cd716cc3d900b81474afa5edf9e92b01aace" exitCode=60 Nov 04 04:00:23.001664 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:23.001645 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv" event={"ID":"a6c5e197-376e-4303-aa47-55057206aa33","Type":"ContainerDied","Data":"a716965fbc9d79f3911e58c9a567cd716cc3d900b81474afa5edf9e92b01aace"} Nov 04 04:00:23.001944 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:23.001705 6000 scope.go:122] "RemoveContainer" containerID="f347bf5344a8bbb8112768ec90b0c29c8e08139a48d927f66b9d97cb3c6dfe26" Nov 04 04:00:23.002166 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:23.002148 6000 scope.go:122] "RemoveContainer" containerID="a716965fbc9d79f3911e58c9a567cd716cc3d900b81474afa5edf9e92b01aace" Nov 04 04:00:23.002512 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:00:23.002485 6000 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv_odh-ai-gateway-infra(a6c5e197-376e-4303-aa47-55057206aa33)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv" podUID="a6c5e197-376e-4303-aa47-55057206aa33" Nov 04 04:00:23.003819 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:23.003797 6000 generic.go:356] "Generic (PLEG): container finished" podID="bf0b6cd3-c2b2-43af-94df-aab3c8d837f5" containerID="03f2d364619d87f3e3b10fb82fb2adb320a11450111998880406f53c962c693b" exitCode=60 Nov 04 04:00:23.003929 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:23.003870 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt" event={"ID":"bf0b6cd3-c2b2-43af-94df-aab3c8d837f5","Type":"ContainerDied","Data":"03f2d364619d87f3e3b10fb82fb2adb320a11450111998880406f53c962c693b"} Nov 04 04:00:23.004213 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:23.004196 6000 scope.go:122] "RemoveContainer" containerID="03f2d364619d87f3e3b10fb82fb2adb320a11450111998880406f53c962c693b" Nov 04 04:00:23.004474 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:00:23.004455 6000 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt_odh-ai-gateway-infra(bf0b6cd3-c2b2-43af-94df-aab3c8d837f5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt" podUID="bf0b6cd3-c2b2-43af-94df-aab3c8d837f5" Nov 04 04:00:23.046741 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:23.045976 6000 scope.go:122] "RemoveContainer" containerID="8e60b1c28298304056a2e45a23a4e119c9b1e5edeb7774a1f3f5b9e8f03f836d" Nov 04 04:00:23.047988 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:23.047966 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv"] Nov 04 04:00:23.070103 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:23.070071 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt"] Nov 04 04:00:23.542985 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:23.542829 6000 patch_prober.go:29] interesting pod/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Nov 04 04:00:23.542985 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:23.542885 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" podUID="599a125f-8022-4f5a-9384-e54dd8e13b32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Nov 04 04:00:24.085026 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:24.084987 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d1426426-d88c-474a-8e0c-b532d0f31756" path="/var/lib/kubelet/pods/d1426426-d88c-474a-8e0c-b532d0f31756/volumes" Nov 04 04:00:24.277779 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:24.275617 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv" Nov 04 04:00:24.360793 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:24.357814 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt" Nov 04 04:00:24.370720 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:24.370687 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a6c5e197-376e-4303-aa47-55057206aa33-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6c5e197-376e-4303-aa47-55057206aa33-maas-api-service-ca\") pod \"a6c5e197-376e-4303-aa47-55057206aa33\" (UID: \"a6c5e197-376e-4303-aa47-55057206aa33\") " Nov 04 04:00:24.370917 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:24.370775 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a6c5e197-376e-4303-aa47-55057206aa33-kube-api-access-dss57\" (UniqueName: \"kubernetes.io/projected/a6c5e197-376e-4303-aa47-55057206aa33-kube-api-access-dss57\") pod \"a6c5e197-376e-4303-aa47-55057206aa33\" (UID: \"a6c5e197-376e-4303-aa47-55057206aa33\") " Nov 04 04:00:24.371073 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:24.371050 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6c5e197-376e-4303-aa47-55057206aa33-maas-api-service-ca" pod "a6c5e197-376e-4303-aa47-55057206aa33" (UID: "a6c5e197-376e-4303-aa47-55057206aa33"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:00:24.371196 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:24.371184 6000 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6c5e197-376e-4303-aa47-55057206aa33-maas-api-service-ca\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:00:24.373035 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:24.373013 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6c5e197-376e-4303-aa47-55057206aa33-kube-api-access-dss57" pod "a6c5e197-376e-4303-aa47-55057206aa33" (UID: "a6c5e197-376e-4303-aa47-55057206aa33"). InnerVolumeSpecName "kube-api-access-dss57". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:00:24.471693 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:24.471652 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bf0b6cd3-c2b2-43af-94df-aab3c8d837f5-kube-api-access-mvzxp\" (UniqueName: \"kubernetes.io/projected/bf0b6cd3-c2b2-43af-94df-aab3c8d837f5-kube-api-access-mvzxp\") pod \"bf0b6cd3-c2b2-43af-94df-aab3c8d837f5\" (UID: \"bf0b6cd3-c2b2-43af-94df-aab3c8d837f5\") " Nov 04 04:00:24.471894 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:24.471871 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bf0b6cd3-c2b2-43af-94df-aab3c8d837f5-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0b6cd3-c2b2-43af-94df-aab3c8d837f5-maas-api-service-ca\") pod \"bf0b6cd3-c2b2-43af-94df-aab3c8d837f5\" (UID: \"bf0b6cd3-c2b2-43af-94df-aab3c8d837f5\") " Nov 04 04:00:24.472158 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:24.472137 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dss57\" (UniqueName: \"kubernetes.io/projected/a6c5e197-376e-4303-aa47-55057206aa33-kube-api-access-dss57\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:00:24.472250 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:24.472188 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf0b6cd3-c2b2-43af-94df-aab3c8d837f5-maas-api-service-ca" pod "bf0b6cd3-c2b2-43af-94df-aab3c8d837f5" (UID: "bf0b6cd3-c2b2-43af-94df-aab3c8d837f5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:00:24.473779 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:24.473741 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf0b6cd3-c2b2-43af-94df-aab3c8d837f5-kube-api-access-mvzxp" pod "bf0b6cd3-c2b2-43af-94df-aab3c8d837f5" (UID: "bf0b6cd3-c2b2-43af-94df-aab3c8d837f5"). InnerVolumeSpecName "kube-api-access-mvzxp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:00:24.542436 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:24.542403 6000 patch_prober.go:29] interesting pod/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Nov 04 04:00:24.542630 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:24.542458 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" podUID="599a125f-8022-4f5a-9384-e54dd8e13b32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Nov 04 04:00:24.573781 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:24.573662 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvzxp\" (UniqueName: \"kubernetes.io/projected/bf0b6cd3-c2b2-43af-94df-aab3c8d837f5-kube-api-access-mvzxp\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:00:24.573781 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:24.573699 6000 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0b6cd3-c2b2-43af-94df-aab3c8d837f5-maas-api-service-ca\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:00:25.054461 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:25.054428 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv" Nov 04 04:00:25.054461 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:25.054450 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv" event={"ID":"a6c5e197-376e-4303-aa47-55057206aa33","Type":"ContainerDied","Data":"32dd71ecd2cfa6fc2d466f04c1c090ebc27272627c3f11b0c5d00ad898680480"} Nov 04 04:00:25.054730 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:25.054523 6000 scope.go:122] "RemoveContainer" containerID="a716965fbc9d79f3911e58c9a567cd716cc3d900b81474afa5edf9e92b01aace" Nov 04 04:00:25.056182 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:25.056154 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt" event={"ID":"bf0b6cd3-c2b2-43af-94df-aab3c8d837f5","Type":"ContainerDied","Data":"d467225f53308d3f762cee5b64f50740d54a498db5264bf6573a4fb284e297a0"} Nov 04 04:00:25.056260 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:25.056231 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt" Nov 04 04:00:25.066742 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:25.066717 6000 scope.go:122] "RemoveContainer" containerID="03f2d364619d87f3e3b10fb82fb2adb320a11450111998880406f53c962c693b" Nov 04 04:00:25.089220 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:25.089190 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv"] Nov 04 04:00:25.101086 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:25.101045 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-36730f-29896080-gsfwv"] Nov 04 04:00:25.138682 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:25.138637 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt"] Nov 04 04:00:25.138957 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:25.138939 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-998997-29896080-lq5dt"] Nov 04 04:00:25.542720 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:25.542680 6000 patch_prober.go:29] interesting pod/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Nov 04 04:00:25.542894 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:25.542742 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" podUID="599a125f-8022-4f5a-9384-e54dd8e13b32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Nov 04 04:00:26.174195 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:26.154605 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a6c5e197-376e-4303-aa47-55057206aa33" path="/var/lib/kubelet/pods/a6c5e197-376e-4303-aa47-55057206aa33/volumes" Nov 04 04:00:26.174195 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:26.155342 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bf0b6cd3-c2b2-43af-94df-aab3c8d837f5" path="/var/lib/kubelet/pods/bf0b6cd3-c2b2-43af-94df-aab3c8d837f5/volumes" Nov 04 04:00:26.544840 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:26.542893 6000 patch_prober.go:29] interesting pod/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Nov 04 04:00:26.544840 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:26.542954 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" podUID="599a125f-8022-4f5a-9384-e54dd8e13b32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Nov 04 04:00:27.544959 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:27.544912 6000 patch_prober.go:29] interesting pod/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Nov 04 04:00:27.545948 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:27.545517 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" podUID="599a125f-8022-4f5a-9384-e54dd8e13b32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Nov 04 04:00:28.542563 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:28.542524 6000 patch_prober.go:29] interesting pod/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Nov 04 04:00:28.542732 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:28.542604 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" podUID="599a125f-8022-4f5a-9384-e54dd8e13b32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Nov 04 04:00:29.548264 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:29.548233 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:29.548774 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:29.548648 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:29.549499 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:29.549474 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:00:35.467328 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.467284 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv"] Nov 04 04:00:35.468195 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.467921 6000 cpu_manager.go:414] "containerMap: removing container" podUID="bf0b6cd3-c2b2-43af-94df-aab3c8d837f5" containerName="cleanup" Nov 04 04:00:35.468195 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.467943 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bf0b6cd3-c2b2-43af-94df-aab3c8d837f5" containerName="cleanup" Nov 04 04:00:35.468195 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.467955 6000 cpu_manager.go:414] "containerMap: removing container" podUID="a6c5e197-376e-4303-aa47-55057206aa33" containerName="cleanup" Nov 04 04:00:35.468195 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.467963 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a6c5e197-376e-4303-aa47-55057206aa33" containerName="cleanup" Nov 04 04:00:35.468195 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.467978 6000 cpu_manager.go:414] "containerMap: removing container" podUID="a6c5e197-376e-4303-aa47-55057206aa33" containerName="cleanup" Nov 04 04:00:35.468195 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.467986 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a6c5e197-376e-4303-aa47-55057206aa33" containerName="cleanup" Nov 04 04:00:35.468195 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.468019 6000 cpu_manager.go:414] "containerMap: removing container" podUID="bf0b6cd3-c2b2-43af-94df-aab3c8d837f5" containerName="cleanup" Nov 04 04:00:35.468195 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.468044 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bf0b6cd3-c2b2-43af-94df-aab3c8d837f5" containerName="cleanup" Nov 04 04:00:35.468195 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.468082 6000 cpu_manager.go:414] "containerMap: removing container" podUID="bf0b6cd3-c2b2-43af-94df-aab3c8d837f5" containerName="cleanup" Nov 04 04:00:35.468195 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.468089 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bf0b6cd3-c2b2-43af-94df-aab3c8d837f5" containerName="cleanup" Nov 04 04:00:35.468195 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.468197 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="bf0b6cd3-c2b2-43af-94df-aab3c8d837f5" containerName="cleanup" Nov 04 04:00:35.469011 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.468211 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="a6c5e197-376e-4303-aa47-55057206aa33" containerName="cleanup" Nov 04 04:00:35.469011 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.468224 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="bf0b6cd3-c2b2-43af-94df-aab3c8d837f5" containerName="cleanup" Nov 04 04:00:35.469011 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.468232 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="a6c5e197-376e-4303-aa47-55057206aa33" containerName="cleanup" Nov 04 04:00:35.469011 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.468239 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="a6c5e197-376e-4303-aa47-55057206aa33" containerName="cleanup" Nov 04 04:00:35.473723 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.473698 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.476318 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.476289 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv"] Nov 04 04:00:35.482483 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.482457 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-64c20d-openshift-default-dockercfg-xpqv6\"" Nov 04 04:00:35.539265 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.539222 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-credential-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.539457 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.539285 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-workload-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.539457 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.539368 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istiod-ca-cert\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.539457 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.539431 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-workload-certs\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.539708 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.539501 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-token\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.539708 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.539565 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvjdr\" (UniqueName: \"kubernetes.io/projected/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-kube-api-access-lvjdr\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.539708 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.539596 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-podinfo\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.539708 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.539654 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-envoy\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.539941 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.539716 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-data\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.568771 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.568722 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd"] Nov 04 04:00:35.569233 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.569219 6000 cpu_manager.go:414] "containerMap: removing container" podUID="a6c5e197-376e-4303-aa47-55057206aa33" containerName="cleanup" Nov 04 04:00:35.569288 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.569234 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a6c5e197-376e-4303-aa47-55057206aa33" containerName="cleanup" Nov 04 04:00:35.569327 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.569319 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="bf0b6cd3-c2b2-43af-94df-aab3c8d837f5" containerName="cleanup" Nov 04 04:00:35.572553 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.572513 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.589502 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.589444 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-4afe0e-openshift-default-dockercfg-x52mt\"" Nov 04 04:00:35.591533 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.591502 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd"] Nov 04 04:00:35.640837 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.640786 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfmcg\" (UniqueName: \"kubernetes.io/projected/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-kube-api-access-hfmcg\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.641036 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.640919 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvjdr\" (UniqueName: \"kubernetes.io/projected/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-kube-api-access-lvjdr\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.641036 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.640974 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-podinfo\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.641145 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.641063 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-envoy\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.641145 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.641121 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-data\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.641145 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.641140 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-workload-certs\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.641290 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.641176 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-credential-socket\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.641290 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.641198 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-credential-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.641290 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.641218 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-podinfo\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.641290 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.641236 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istiod-ca-cert\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.641290 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.641258 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-data\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.641290 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.641286 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-envoy\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.641595 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.641306 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-workload-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.641595 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.641326 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istiod-ca-cert\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.641595 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.641510 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-workload-socket\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.641740 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.641599 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-data\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.641740 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.641642 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-credential-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.641740 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.641670 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-workload-certs\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.641740 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.641729 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-token\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.641985 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.641800 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-token\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.641985 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.641885 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-workload-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.641985 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.641950 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-workload-certs\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.642139 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.642062 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istiod-ca-cert\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.643699 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.643667 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-podinfo\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.644244 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.644222 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-envoy\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.658006 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.657973 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-token\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.658206 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.658182 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvjdr\" (UniqueName: \"kubernetes.io/projected/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-kube-api-access-lvjdr\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.743552 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.743452 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfmcg\" (UniqueName: \"kubernetes.io/projected/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-kube-api-access-hfmcg\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.743742 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.743554 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-workload-certs\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.743742 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.743589 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-credential-socket\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.743742 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.743616 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-podinfo\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.743742 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.743643 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istiod-ca-cert\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.743742 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.743672 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-data\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.743742 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.743711 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-envoy\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.744084 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.743747 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-workload-socket\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.744084 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.743817 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-token\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.744182 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.744075 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-credential-socket\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.744182 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.744098 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-workload-certs\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.744182 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.744138 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-data\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.744380 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.744357 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-workload-socket\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.744768 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.744731 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istiod-ca-cert\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.746297 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.746271 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-envoy\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.746585 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.746568 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-podinfo\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.756479 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.756228 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-token\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.756678 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.756651 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfmcg\" (UniqueName: \"kubernetes.io/projected/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-kube-api-access-hfmcg\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:35.792336 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.792305 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:35.889445 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:35.889396 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:36.083615 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:36.083582 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv"] Nov 04 04:00:36.086825 ip-10-0-129-134 kubenswrapper[6000]: W1104 04:00:36.086794 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8354d1d_fe79_42ca_b453_e8b315d4b9a8.slice/crio-b438ca74e59b90e51bcd28ae1893c7c177eaaeb5048491538b64e8f2aa3e24a9 WatchSource:0}: Error finding container b438ca74e59b90e51bcd28ae1893c7c177eaaeb5048491538b64e8f2aa3e24a9: Status 404 returned error can't find the container with id b438ca74e59b90e51bcd28ae1893c7c177eaaeb5048491538b64e8f2aa3e24a9 Nov 04 04:00:36.089176 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:36.089144 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:00:36.089262 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:36.089214 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:00:36.089262 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:36.089243 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:00:36.130562 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:36.130519 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" event={"ID":"a8354d1d-fe79-42ca-b453-e8b315d4b9a8","Type":"ContainerStarted","Data":"b438ca74e59b90e51bcd28ae1893c7c177eaaeb5048491538b64e8f2aa3e24a9"} Nov 04 04:00:36.183246 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:36.183216 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd"] Nov 04 04:00:36.185457 ip-10-0-129-134 kubenswrapper[6000]: W1104 04:00:36.185425 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f6ecbe7_6df4_43c3_b913_9dfd6e1f79cd.slice/crio-e6e6865eed807a04dc9f6d07cdc50f671f121ec42dbcb782e1ea75d12992de90 WatchSource:0}: Error finding container e6e6865eed807a04dc9f6d07cdc50f671f121ec42dbcb782e1ea75d12992de90: Status 404 returned error can't find the container with id e6e6865eed807a04dc9f6d07cdc50f671f121ec42dbcb782e1ea75d12992de90 Nov 04 04:00:36.188234 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:36.188191 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:00:36.188346 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:36.188289 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:00:36.188346 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:36.188340 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:00:37.176581 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:37.176540 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" event={"ID":"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd","Type":"ContainerStarted","Data":"c9e03e31be5c0d4e6fd3e9a916023c401f51584ffc1c1b749006e764f48adfa0"} Nov 04 04:00:37.177130 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:37.176605 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" event={"ID":"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd","Type":"ContainerStarted","Data":"e6e6865eed807a04dc9f6d07cdc50f671f121ec42dbcb782e1ea75d12992de90"} Nov 04 04:00:37.178537 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:37.178509 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" event={"ID":"a8354d1d-fe79-42ca-b453-e8b315d4b9a8","Type":"ContainerStarted","Data":"7400917a30dea2e8b404edff1f038799c72a46b0174da8545b6db2750a64f8a5"} Nov 04 04:00:37.202701 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:37.202649 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" podStartSLOduration=2.202633598 podStartE2EDuration="2.202633598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:00:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:00:37.201674363 +0000 UTC m=+635.240114328" watchObservedRunningTime="2026-11-04 04:00:37.202633598 +0000 UTC m=+635.241073561" Nov 04 04:00:37.228785 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:37.227544 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" podStartSLOduration=2.227525612 podStartE2EDuration="2.227525612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:00:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:00:37.227228035 +0000 UTC m=+635.265667999" watchObservedRunningTime="2026-11-04 04:00:37.227525612 +0000 UTC m=+635.265965577" Nov 04 04:00:37.792716 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:37.792670 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:37.794288 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:37.794263 6000 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Nov 04 04:00:37.794374 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:37.794317 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" podUID="a8354d1d-fe79-42ca-b453-e8b315d4b9a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Nov 04 04:00:37.890782 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:37.890054 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:37.896174 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:37.894696 6000 patch_prober.go:29] interesting pod/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Nov 04 04:00:37.896174 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:37.894780 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" podUID="2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Nov 04 04:00:38.769139 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:38.769101 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-zf6m4"] Nov 04 04:00:38.793373 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:38.793331 6000 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Nov 04 04:00:38.793613 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:38.793593 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" podUID="a8354d1d-fe79-42ca-b453-e8b315d4b9a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Nov 04 04:00:38.890501 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:38.890466 6000 patch_prober.go:29] interesting pod/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Nov 04 04:00:38.890670 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:38.890533 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" podUID="2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Nov 04 04:00:39.800858 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:39.796893 6000 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Nov 04 04:00:39.800858 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:39.796965 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" podUID="a8354d1d-fe79-42ca-b453-e8b315d4b9a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Nov 04 04:00:39.893581 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:39.893544 6000 patch_prober.go:29] interesting pod/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Nov 04 04:00:39.893908 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:39.893880 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" podUID="2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Nov 04 04:00:40.792887 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:40.792839 6000 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Nov 04 04:00:40.793092 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:40.792905 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" podUID="a8354d1d-fe79-42ca-b453-e8b315d4b9a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Nov 04 04:00:40.890412 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:40.890374 6000 patch_prober.go:29] interesting pod/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Nov 04 04:00:40.890963 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:40.890433 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" podUID="2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Nov 04 04:00:41.793440 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:41.793152 6000 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Nov 04 04:00:41.793440 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:41.793238 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" podUID="a8354d1d-fe79-42ca-b453-e8b315d4b9a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Nov 04 04:00:41.890138 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:41.890108 6000 patch_prober.go:29] interesting pod/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Nov 04 04:00:41.890355 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:41.890333 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" podUID="2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Nov 04 04:00:42.793704 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:42.793664 6000 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Nov 04 04:00:42.794115 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:42.793739 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" podUID="a8354d1d-fe79-42ca-b453-e8b315d4b9a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Nov 04 04:00:42.890141 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:42.890095 6000 patch_prober.go:29] interesting pod/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Nov 04 04:00:42.890367 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:42.890163 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" podUID="2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Nov 04 04:00:43.797703 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:43.797670 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:43.798219 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:43.797968 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:43.798980 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:43.798960 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:00:43.894835 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:43.894801 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:43.895225 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:43.895194 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:43.896208 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:43.896185 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:00:50.751728 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:00:50.751685 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 04:00:54.898481 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:00:54.898446 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-zf6m4"] Nov 04 04:01:01.696073 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:01.696034 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz"] Nov 04 04:01:01.701566 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:01.701530 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65"] Nov 04 04:01:10.688592 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:10.688555 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s"] Nov 04 04:01:15.689130 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:15.689044 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s" podUID="52e771b7-2c73-4b2f-8266-f37cb72490bd" containerName="payload-pre-processing" containerID="cri-o://222676571a7b3bc0c14c3ee89cc468fc83b83d000a9a74d70b56d8567a73a7c4" gracePeriod=25 Nov 04 04:01:16.722009 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:16.721962 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" podUID="4d06ff75-d1d7-440e-98fe-90fd6045618d" containerName="main" containerID="cri-o://088f28147ea57978d96c7cf6c782acaeb0600835ba24d9c072418d651b4424f5" gracePeriod=45 Nov 04 04:01:16.723124 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:16.722724 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" podUID="9db4519d-0989-45cc-9198-22f41c15c951" containerName="main" containerID="cri-o://51f18bdc4ebccfb04caf6e7664cb8b6057eef543e0590d5130765f8f9a4a5a5d" gracePeriod=45 Nov 04 04:01:17.005969 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.005939 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 04:01:17.019398 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.019364 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-model-cache\") pod \"4d06ff75-d1d7-440e-98fe-90fd6045618d\" (UID: \"4d06ff75-d1d7-440e-98fe-90fd6045618d\") " Nov 04 04:01:17.019568 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.019434 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-home\" (UniqueName: \"kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-home\") pod \"4d06ff75-d1d7-440e-98fe-90fd6045618d\" (UID: \"4d06ff75-d1d7-440e-98fe-90fd6045618d\") " Nov 04 04:01:17.019568 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.019491 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-tmp-dir\") pod \"4d06ff75-d1d7-440e-98fe-90fd6045618d\" (UID: \"4d06ff75-d1d7-440e-98fe-90fd6045618d\") " Nov 04 04:01:17.019568 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.019550 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4d06ff75-d1d7-440e-98fe-90fd6045618d-tls-certs\" (UniqueName: \"kubernetes.io/secret/4d06ff75-d1d7-440e-98fe-90fd6045618d-tls-certs\") pod \"4d06ff75-d1d7-440e-98fe-90fd6045618d\" (UID: \"4d06ff75-d1d7-440e-98fe-90fd6045618d\") " Nov 04 04:01:17.019733 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.019697 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-dshm\") pod \"4d06ff75-d1d7-440e-98fe-90fd6045618d\" (UID: \"4d06ff75-d1d7-440e-98fe-90fd6045618d\") " Nov 04 04:01:17.019733 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.019726 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d06ff75-d1d7-440e-98fe-90fd6045618d-kube-api-access-fsnbc\" (UniqueName: \"kubernetes.io/projected/4d06ff75-d1d7-440e-98fe-90fd6045618d-kube-api-access-fsnbc\") pod \"4d06ff75-d1d7-440e-98fe-90fd6045618d\" (UID: \"4d06ff75-d1d7-440e-98fe-90fd6045618d\") " Nov 04 04:01:17.023994 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.020613 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-model-cache" pod "4d06ff75-d1d7-440e-98fe-90fd6045618d" (UID: "4d06ff75-d1d7-440e-98fe-90fd6045618d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:17.023994 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.020867 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-tmp-dir" pod "4d06ff75-d1d7-440e-98fe-90fd6045618d" (UID: "4d06ff75-d1d7-440e-98fe-90fd6045618d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:17.023994 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.020995 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-home" pod "4d06ff75-d1d7-440e-98fe-90fd6045618d" (UID: "4d06ff75-d1d7-440e-98fe-90fd6045618d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:17.023994 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.023037 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d06ff75-d1d7-440e-98fe-90fd6045618d-kube-api-access-fsnbc" pod "4d06ff75-d1d7-440e-98fe-90fd6045618d" (UID: "4d06ff75-d1d7-440e-98fe-90fd6045618d"). InnerVolumeSpecName "kube-api-access-fsnbc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:01:17.023994 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.023685 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d06ff75-d1d7-440e-98fe-90fd6045618d-tls-certs" pod "4d06ff75-d1d7-440e-98fe-90fd6045618d" (UID: "4d06ff75-d1d7-440e-98fe-90fd6045618d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 04:01:17.024915 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.024891 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-dshm" pod "4d06ff75-d1d7-440e-98fe-90fd6045618d" (UID: "4d06ff75-d1d7-440e-98fe-90fd6045618d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:17.034234 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.034206 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 04:01:17.121188 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.121151 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-model-cache\") pod \"9db4519d-0989-45cc-9198-22f41c15c951\" (UID: \"9db4519d-0989-45cc-9198-22f41c15c951\") " Nov 04 04:01:17.121383 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.121260 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-tmp-dir\") pod \"9db4519d-0989-45cc-9198-22f41c15c951\" (UID: \"9db4519d-0989-45cc-9198-22f41c15c951\") " Nov 04 04:01:17.121383 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.121291 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9db4519d-0989-45cc-9198-22f41c15c951-tls-certs\" (UniqueName: \"kubernetes.io/secret/9db4519d-0989-45cc-9198-22f41c15c951-tls-certs\") pod \"9db4519d-0989-45cc-9198-22f41c15c951\" (UID: \"9db4519d-0989-45cc-9198-22f41c15c951\") " Nov 04 04:01:17.121383 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.121319 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9db4519d-0989-45cc-9198-22f41c15c951-kube-api-access-h7vlp\" (UniqueName: \"kubernetes.io/projected/9db4519d-0989-45cc-9198-22f41c15c951-kube-api-access-h7vlp\") pod \"9db4519d-0989-45cc-9198-22f41c15c951\" (UID: \"9db4519d-0989-45cc-9198-22f41c15c951\") " Nov 04 04:01:17.121505 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.121432 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-dshm\") pod \"9db4519d-0989-45cc-9198-22f41c15c951\" (UID: \"9db4519d-0989-45cc-9198-22f41c15c951\") " Nov 04 04:01:17.121505 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.121458 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-home\" (UniqueName: \"kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-home\") pod \"9db4519d-0989-45cc-9198-22f41c15c951\" (UID: \"9db4519d-0989-45cc-9198-22f41c15c951\") " Nov 04 04:01:17.121566 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.121494 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-model-cache" pod "9db4519d-0989-45cc-9198-22f41c15c951" (UID: "9db4519d-0989-45cc-9198-22f41c15c951"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:17.121604 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.121557 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-tmp-dir" pod "9db4519d-0989-45cc-9198-22f41c15c951" (UID: "9db4519d-0989-45cc-9198-22f41c15c951"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:17.121990 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.121964 6000 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-model-cache\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:17.122106 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.121992 6000 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-home\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:17.122106 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.122008 6000 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-model-cache\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:17.122106 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.122021 6000 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-tmp-dir\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:17.122106 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.122034 6000 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d06ff75-d1d7-440e-98fe-90fd6045618d-tls-certs\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:17.122106 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.122047 6000 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-tmp-dir\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:17.122106 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.121967 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-home" pod "9db4519d-0989-45cc-9198-22f41c15c951" (UID: "9db4519d-0989-45cc-9198-22f41c15c951"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:17.122106 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.122061 6000 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d06ff75-d1d7-440e-98fe-90fd6045618d-dshm\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:17.122106 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.122100 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsnbc\" (UniqueName: \"kubernetes.io/projected/4d06ff75-d1d7-440e-98fe-90fd6045618d-kube-api-access-fsnbc\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:17.123534 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.123503 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9db4519d-0989-45cc-9198-22f41c15c951-tls-certs" pod "9db4519d-0989-45cc-9198-22f41c15c951" (UID: "9db4519d-0989-45cc-9198-22f41c15c951"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 04:01:17.123724 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.123700 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-dshm" pod "9db4519d-0989-45cc-9198-22f41c15c951" (UID: "9db4519d-0989-45cc-9198-22f41c15c951"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:17.124158 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.124139 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9db4519d-0989-45cc-9198-22f41c15c951-kube-api-access-h7vlp" pod "9db4519d-0989-45cc-9198-22f41c15c951" (UID: "9db4519d-0989-45cc-9198-22f41c15c951"). InnerVolumeSpecName "kube-api-access-h7vlp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:01:17.222727 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.222692 6000 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-dshm\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:17.222727 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.222725 6000 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9db4519d-0989-45cc-9198-22f41c15c951-home\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:17.222727 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.222734 6000 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9db4519d-0989-45cc-9198-22f41c15c951-tls-certs\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:17.223001 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.222745 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7vlp\" (UniqueName: \"kubernetes.io/projected/9db4519d-0989-45cc-9198-22f41c15c951-kube-api-access-h7vlp\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:17.592816 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.592750 6000 generic.go:356] "Generic (PLEG): container finished" podID="9db4519d-0989-45cc-9198-22f41c15c951" containerID="51f18bdc4ebccfb04caf6e7664cb8b6057eef543e0590d5130765f8f9a4a5a5d" exitCode=0 Nov 04 04:01:17.593129 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.592851 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" Nov 04 04:01:17.593278 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.592862 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" event={"ID":"9db4519d-0989-45cc-9198-22f41c15c951","Type":"ContainerDied","Data":"51f18bdc4ebccfb04caf6e7664cb8b6057eef543e0590d5130765f8f9a4a5a5d"} Nov 04 04:01:17.593278 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.593264 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65" event={"ID":"9db4519d-0989-45cc-9198-22f41c15c951","Type":"ContainerDied","Data":"115fdc52b020ad77551f7a1be2a080a55e49f3f5a1acbc17eb656a1161b59316"} Nov 04 04:01:17.593404 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.593304 6000 scope.go:122] "RemoveContainer" containerID="51f18bdc4ebccfb04caf6e7664cb8b6057eef543e0590d5130765f8f9a4a5a5d" Nov 04 04:01:17.594908 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.594878 6000 generic.go:356] "Generic (PLEG): container finished" podID="4d06ff75-d1d7-440e-98fe-90fd6045618d" containerID="088f28147ea57978d96c7cf6c782acaeb0600835ba24d9c072418d651b4424f5" exitCode=0 Nov 04 04:01:17.595056 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.594924 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" event={"ID":"4d06ff75-d1d7-440e-98fe-90fd6045618d","Type":"ContainerDied","Data":"088f28147ea57978d96c7cf6c782acaeb0600835ba24d9c072418d651b4424f5"} Nov 04 04:01:17.595056 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.594956 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" event={"ID":"4d06ff75-d1d7-440e-98fe-90fd6045618d","Type":"ContainerDied","Data":"b1ef72aa811cbf5c34271d2a3c0e4a156b1b58b2943b8c01bd4a5af6030c5907"} Nov 04 04:01:17.595056 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.595020 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz" Nov 04 04:01:17.606071 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.606046 6000 scope.go:122] "RemoveContainer" containerID="51f18bdc4ebccfb04caf6e7664cb8b6057eef543e0590d5130765f8f9a4a5a5d" Nov 04 04:01:17.606488 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:01:17.606465 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51f18bdc4ebccfb04caf6e7664cb8b6057eef543e0590d5130765f8f9a4a5a5d\": container with ID starting with 51f18bdc4ebccfb04caf6e7664cb8b6057eef543e0590d5130765f8f9a4a5a5d not found: ID does not exist" containerID="51f18bdc4ebccfb04caf6e7664cb8b6057eef543e0590d5130765f8f9a4a5a5d" Nov 04 04:01:17.606651 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.606624 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51f18bdc4ebccfb04caf6e7664cb8b6057eef543e0590d5130765f8f9a4a5a5d"} err="failed to get container status \"51f18bdc4ebccfb04caf6e7664cb8b6057eef543e0590d5130765f8f9a4a5a5d\": rpc error: code = NotFound desc = could not find container \"51f18bdc4ebccfb04caf6e7664cb8b6057eef543e0590d5130765f8f9a4a5a5d\": container with ID starting with 51f18bdc4ebccfb04caf6e7664cb8b6057eef543e0590d5130765f8f9a4a5a5d not found: ID does not exist" Nov 04 04:01:17.606822 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.606807 6000 scope.go:122] "RemoveContainer" containerID="088f28147ea57978d96c7cf6c782acaeb0600835ba24d9c072418d651b4424f5" Nov 04 04:01:17.617134 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.617106 6000 scope.go:122] "RemoveContainer" containerID="088f28147ea57978d96c7cf6c782acaeb0600835ba24d9c072418d651b4424f5" Nov 04 04:01:17.617558 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:01:17.617521 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"088f28147ea57978d96c7cf6c782acaeb0600835ba24d9c072418d651b4424f5\": container with ID starting with 088f28147ea57978d96c7cf6c782acaeb0600835ba24d9c072418d651b4424f5 not found: ID does not exist" containerID="088f28147ea57978d96c7cf6c782acaeb0600835ba24d9c072418d651b4424f5" Nov 04 04:01:17.617653 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.617558 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"088f28147ea57978d96c7cf6c782acaeb0600835ba24d9c072418d651b4424f5"} err="failed to get container status \"088f28147ea57978d96c7cf6c782acaeb0600835ba24d9c072418d651b4424f5\": rpc error: code = NotFound desc = could not find container \"088f28147ea57978d96c7cf6c782acaeb0600835ba24d9c072418d651b4424f5\": container with ID starting with 088f28147ea57978d96c7cf6c782acaeb0600835ba24d9c072418d651b4424f5 not found: ID does not exist" Nov 04 04:01:17.623146 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.623111 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65"] Nov 04 04:01:17.630725 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.630688 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-78ca37/simulator-w2-78ca37-kserve-5fd8887974-vxv65"] Nov 04 04:01:17.642189 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.642144 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz"] Nov 04 04:01:17.646088 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:17.646051 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-78ca37/premium-w2-78ca37-kserve-97c65cff5-9wfzz"] Nov 04 04:01:18.083509 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:18.083467 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4d06ff75-d1d7-440e-98fe-90fd6045618d" path="/var/lib/kubelet/pods/4d06ff75-d1d7-440e-98fe-90fd6045618d/volumes" Nov 04 04:01:18.083997 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:18.083931 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9db4519d-0989-45cc-9198-22f41c15c951" path="/var/lib/kubelet/pods/9db4519d-0989-45cc-9198-22f41c15c951/volumes" Nov 04 04:01:20.022546 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:20.022496 6000 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s" podUID="52e771b7-2c73-4b2f-8266-f37cb72490bd" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 04 04:01:20.753473 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:01:20.753428 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 04:01:24.312387 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:24.312344 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-zf6m4"] Nov 04 04:01:26.098780 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:26.097820 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg"] Nov 04 04:01:26.109780 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:26.104701 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr"] Nov 04 04:01:30.023104 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:30.023052 6000 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s" podUID="52e771b7-2c73-4b2f-8266-f37cb72490bd" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 04 04:01:31.966799 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:31.950019 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l"] Nov 04 04:01:32.569154 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:32.569117 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4"] Nov 04 04:01:32.569416 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:32.569364 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4" podUID="5195f539-2ed2-45c8-bdbe-51d098e321b8" containerName="maas-api" containerID="cri-o://874a541b4b75600e526259188c85ef943bf964361a00ad65cbe0811aeec37d14" gracePeriod=30 Nov 04 04:01:33.109651 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:33.108005 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4" Nov 04 04:01:33.192144 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:33.192112 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5195f539-2ed2-45c8-bdbe-51d098e321b8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5195f539-2ed2-45c8-bdbe-51d098e321b8-metrics-certs\") pod \"5195f539-2ed2-45c8-bdbe-51d098e321b8\" (UID: \"5195f539-2ed2-45c8-bdbe-51d098e321b8\") " Nov 04 04:01:33.192335 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:33.192171 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5195f539-2ed2-45c8-bdbe-51d098e321b8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5195f539-2ed2-45c8-bdbe-51d098e321b8-maas-api-tls\") pod \"5195f539-2ed2-45c8-bdbe-51d098e321b8\" (UID: \"5195f539-2ed2-45c8-bdbe-51d098e321b8\") " Nov 04 04:01:33.192335 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:33.192204 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5195f539-2ed2-45c8-bdbe-51d098e321b8-kube-api-access-dsg57\" (UniqueName: \"kubernetes.io/projected/5195f539-2ed2-45c8-bdbe-51d098e321b8-kube-api-access-dsg57\") pod \"5195f539-2ed2-45c8-bdbe-51d098e321b8\" (UID: \"5195f539-2ed2-45c8-bdbe-51d098e321b8\") " Nov 04 04:01:33.192335 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:33.192293 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5195f539-2ed2-45c8-bdbe-51d098e321b8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5195f539-2ed2-45c8-bdbe-51d098e321b8-openshift-service-ca\") pod \"5195f539-2ed2-45c8-bdbe-51d098e321b8\" (UID: \"5195f539-2ed2-45c8-bdbe-51d098e321b8\") " Nov 04 04:01:33.192727 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:33.192688 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5195f539-2ed2-45c8-bdbe-51d098e321b8-openshift-service-ca" pod "5195f539-2ed2-45c8-bdbe-51d098e321b8" (UID: "5195f539-2ed2-45c8-bdbe-51d098e321b8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:01:33.194482 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:33.194454 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5195f539-2ed2-45c8-bdbe-51d098e321b8-maas-api-tls" pod "5195f539-2ed2-45c8-bdbe-51d098e321b8" (UID: "5195f539-2ed2-45c8-bdbe-51d098e321b8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 04:01:33.194791 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:33.194745 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5195f539-2ed2-45c8-bdbe-51d098e321b8-metrics-certs" pod "5195f539-2ed2-45c8-bdbe-51d098e321b8" (UID: "5195f539-2ed2-45c8-bdbe-51d098e321b8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 04:01:33.194791 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:33.194782 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5195f539-2ed2-45c8-bdbe-51d098e321b8-kube-api-access-dsg57" pod "5195f539-2ed2-45c8-bdbe-51d098e321b8" (UID: "5195f539-2ed2-45c8-bdbe-51d098e321b8"). InnerVolumeSpecName "kube-api-access-dsg57". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:01:33.294060 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:33.294027 6000 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5195f539-2ed2-45c8-bdbe-51d098e321b8-metrics-certs\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:33.294060 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:33.294055 6000 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5195f539-2ed2-45c8-bdbe-51d098e321b8-maas-api-tls\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:33.294060 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:33.294066 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsg57\" (UniqueName: \"kubernetes.io/projected/5195f539-2ed2-45c8-bdbe-51d098e321b8-kube-api-access-dsg57\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:33.294301 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:33.294076 6000 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5195f539-2ed2-45c8-bdbe-51d098e321b8-openshift-service-ca\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:33.735587 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:33.735550 6000 generic.go:356] "Generic (PLEG): container finished" podID="5195f539-2ed2-45c8-bdbe-51d098e321b8" containerID="874a541b4b75600e526259188c85ef943bf964361a00ad65cbe0811aeec37d14" exitCode=0 Nov 04 04:01:33.735794 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:33.735605 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4" Nov 04 04:01:33.735794 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:33.735641 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4" event={"ID":"5195f539-2ed2-45c8-bdbe-51d098e321b8","Type":"ContainerDied","Data":"874a541b4b75600e526259188c85ef943bf964361a00ad65cbe0811aeec37d14"} Nov 04 04:01:33.735794 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:33.735705 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4" event={"ID":"5195f539-2ed2-45c8-bdbe-51d098e321b8","Type":"ContainerDied","Data":"41279bb6bda46d9c502cf7f4857d9ad4e47a36e2ce08399875e6928067915ae8"} Nov 04 04:01:33.735794 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:33.735734 6000 scope.go:122] "RemoveContainer" containerID="874a541b4b75600e526259188c85ef943bf964361a00ad65cbe0811aeec37d14" Nov 04 04:01:33.756782 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:33.755146 6000 scope.go:122] "RemoveContainer" containerID="874a541b4b75600e526259188c85ef943bf964361a00ad65cbe0811aeec37d14" Nov 04 04:01:33.756782 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:01:33.755984 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"874a541b4b75600e526259188c85ef943bf964361a00ad65cbe0811aeec37d14\": container with ID starting with 874a541b4b75600e526259188c85ef943bf964361a00ad65cbe0811aeec37d14 not found: ID does not exist" containerID="874a541b4b75600e526259188c85ef943bf964361a00ad65cbe0811aeec37d14" Nov 04 04:01:33.756782 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:33.756033 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"874a541b4b75600e526259188c85ef943bf964361a00ad65cbe0811aeec37d14"} err="failed to get container status \"874a541b4b75600e526259188c85ef943bf964361a00ad65cbe0811aeec37d14\": rpc error: code = NotFound desc = could not find container \"874a541b4b75600e526259188c85ef943bf964361a00ad65cbe0811aeec37d14\": container with ID starting with 874a541b4b75600e526259188c85ef943bf964361a00ad65cbe0811aeec37d14 not found: ID does not exist" Nov 04 04:01:33.774292 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:33.774253 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4"] Nov 04 04:01:33.780436 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:33.780397 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-6a9cdb-675bcf9998-vmbx4"] Nov 04 04:01:34.087452 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:34.087408 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5195f539-2ed2-45c8-bdbe-51d098e321b8" path="/var/lib/kubelet/pods/5195f539-2ed2-45c8-bdbe-51d098e321b8/volumes" Nov 04 04:01:34.452832 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:34.451683 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc"] Nov 04 04:01:34.452832 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:34.451966 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" podUID="48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d" containerName="istio-proxy" containerID="cri-o://c7d765357c5924ff5a248cb7e9f029accb6d15fc803813ec2214e341b8a636d8" gracePeriod=30 Nov 04 04:01:35.594914 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.594876 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s"] Nov 04 04:01:35.596490 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.596315 6000 cpu_manager.go:414] "containerMap: removing container" podUID="9db4519d-0989-45cc-9198-22f41c15c951" containerName="main" Nov 04 04:01:35.596792 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.596724 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9db4519d-0989-45cc-9198-22f41c15c951" containerName="main" Nov 04 04:01:35.596893 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.596796 6000 cpu_manager.go:414] "containerMap: removing container" podUID="4d06ff75-d1d7-440e-98fe-90fd6045618d" containerName="main" Nov 04 04:01:35.596893 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.596807 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d06ff75-d1d7-440e-98fe-90fd6045618d" containerName="main" Nov 04 04:01:35.597010 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.596912 6000 cpu_manager.go:414] "containerMap: removing container" podUID="5195f539-2ed2-45c8-bdbe-51d098e321b8" containerName="maas-api" Nov 04 04:01:35.597010 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.596922 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5195f539-2ed2-45c8-bdbe-51d098e321b8" containerName="maas-api" Nov 04 04:01:35.597122 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.597071 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="4d06ff75-d1d7-440e-98fe-90fd6045618d" containerName="main" Nov 04 04:01:35.597122 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.597091 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="5195f539-2ed2-45c8-bdbe-51d098e321b8" containerName="maas-api" Nov 04 04:01:35.597122 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.597104 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="9db4519d-0989-45cc-9198-22f41c15c951" containerName="main" Nov 04 04:01:35.602628 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.602597 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.632241 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.632050 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s"] Nov 04 04:01:35.760407 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.760373 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-data\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.760407 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.760415 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-workload-certs\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.760678 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.760529 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-credential-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.760678 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.760590 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-envoy\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.760809 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.760677 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-workload-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.760809 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.760738 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5chw\" (UniqueName: \"kubernetes.io/projected/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-kube-api-access-s5chw\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.760920 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.760817 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-token\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.760920 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.760841 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istiod-ca-cert\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.760920 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.760858 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-podinfo\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.801529 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.801494 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c"] Nov 04 04:01:35.806542 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.806514 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:35.822994 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.819792 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c"] Nov 04 04:01:35.867112 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.867017 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-credential-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.867112 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.867093 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-envoy\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.867329 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.867156 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-workload-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.867329 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.867245 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5chw\" (UniqueName: \"kubernetes.io/projected/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-kube-api-access-s5chw\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.867329 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.867269 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-token\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.867329 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.867300 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istiod-ca-cert\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.867329 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.867322 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-podinfo\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.867544 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.867347 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-data\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.867544 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.867389 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-workload-certs\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.867866 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.867840 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-workload-certs\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.868135 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.868113 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-workload-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.868244 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.868135 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-data\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.868568 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.868547 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istiod-ca-cert\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.868708 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.868690 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-credential-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.871062 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.871030 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-envoy\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.871209 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.871191 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-podinfo\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.880528 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.880461 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-token\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.880688 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.880590 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5chw\" (UniqueName: \"kubernetes.io/projected/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-kube-api-access-s5chw\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.945092 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.944811 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:35.969003 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.968962 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-workload-certs\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:35.969240 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.969019 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-podinfo\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:35.969240 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.969165 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bxmx\" (UniqueName: \"kubernetes.io/projected/78a5d7c4-f484-4788-a260-ead9c2ba30da-kube-api-access-9bxmx\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:35.969240 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.969226 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-data\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:35.969500 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.969313 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78a5d7c4-f484-4788-a260-ead9c2ba30da-istiod-ca-cert\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:35.969500 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.969402 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-credential-socket\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:35.969500 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.969434 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-envoy\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:35.969500 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.969488 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-token\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:35.969862 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.969518 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-workload-socket\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:35.987357 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:35.987326 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s" Nov 04 04:01:36.070911 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.070867 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52e771b7-2c73-4b2f-8266-f37cb72490bd-kube-api-access-9z2vh\" (UniqueName: \"kubernetes.io/projected/52e771b7-2c73-4b2f-8266-f37cb72490bd-kube-api-access-9z2vh\") pod \"52e771b7-2c73-4b2f-8266-f37cb72490bd\" (UID: \"52e771b7-2c73-4b2f-8266-f37cb72490bd\") " Nov 04 04:01:36.071114 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.070954 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/52e771b7-2c73-4b2f-8266-f37cb72490bd-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52e771b7-2c73-4b2f-8266-f37cb72490bd-plugins-config-volume\") pod \"52e771b7-2c73-4b2f-8266-f37cb72490bd\" (UID: \"52e771b7-2c73-4b2f-8266-f37cb72490bd\") " Nov 04 04:01:36.071114 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.071081 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-workload-certs\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:36.071114 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.071102 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-podinfo\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:36.071299 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.071134 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bxmx\" (UniqueName: \"kubernetes.io/projected/78a5d7c4-f484-4788-a260-ead9c2ba30da-kube-api-access-9bxmx\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:36.071299 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.071158 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-data\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:36.071299 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.071207 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78a5d7c4-f484-4788-a260-ead9c2ba30da-istiod-ca-cert\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:36.071299 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.071256 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-credential-socket\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:36.071299 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.071276 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-envoy\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:36.071547 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.071311 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-token\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:36.071547 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.071329 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-workload-socket\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:36.071711 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.071691 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-workload-socket\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:36.072310 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.072286 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-data\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:36.072497 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.072477 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-credential-socket\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:36.072598 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.072585 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-workload-certs\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:36.072661 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.072624 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52e771b7-2c73-4b2f-8266-f37cb72490bd-plugins-config-volume" pod "52e771b7-2c73-4b2f-8266-f37cb72490bd" (UID: "52e771b7-2c73-4b2f-8266-f37cb72490bd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:01:36.072951 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.072932 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78a5d7c4-f484-4788-a260-ead9c2ba30da-istiod-ca-cert\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:36.074143 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.074111 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52e771b7-2c73-4b2f-8266-f37cb72490bd-kube-api-access-9z2vh" pod "52e771b7-2c73-4b2f-8266-f37cb72490bd" (UID: "52e771b7-2c73-4b2f-8266-f37cb72490bd"). InnerVolumeSpecName "kube-api-access-9z2vh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:01:36.075197 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.075150 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-envoy\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:36.076303 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.076280 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-podinfo\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:36.084786 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.084711 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-token\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:36.085546 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.085522 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bxmx\" (UniqueName: \"kubernetes.io/projected/78a5d7c4-f484-4788-a260-ead9c2ba30da-kube-api-access-9bxmx\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:36.136124 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.136028 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:36.172912 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.172870 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9z2vh\" (UniqueName: \"kubernetes.io/projected/52e771b7-2c73-4b2f-8266-f37cb72490bd-kube-api-access-9z2vh\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:36.173077 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.172927 6000 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52e771b7-2c73-4b2f-8266-f37cb72490bd-plugins-config-volume\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:36.244855 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.244811 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s"] Nov 04 04:01:36.251262 ip-10-0-129-134 kubenswrapper[6000]: W1104 04:01:36.251235 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4afd26c6_87c7_4e7e_a315_5e4106ce8a21.slice/crio-e092052b57012ed2d8d76062b4dc4a2a1156222a79b287536ce971025e851be7 WatchSource:0}: Error finding container e092052b57012ed2d8d76062b4dc4a2a1156222a79b287536ce971025e851be7: Status 404 returned error can't find the container with id e092052b57012ed2d8d76062b4dc4a2a1156222a79b287536ce971025e851be7 Nov 04 04:01:36.254206 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.253950 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:01:36.254326 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.254273 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:01:36.254384 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.254327 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:01:36.426729 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.426596 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c"] Nov 04 04:01:36.431463 ip-10-0-129-134 kubenswrapper[6000]: W1104 04:01:36.431434 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78a5d7c4_f484_4788_a260_ead9c2ba30da.slice/crio-edfdfc10a69c7c6e84faeccbc6323b5c15cab445449941b0a763ca32eddc190d WatchSource:0}: Error finding container edfdfc10a69c7c6e84faeccbc6323b5c15cab445449941b0a763ca32eddc190d: Status 404 returned error can't find the container with id edfdfc10a69c7c6e84faeccbc6323b5c15cab445449941b0a763ca32eddc190d Nov 04 04:01:36.434506 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.434467 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:01:36.434623 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.434538 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:01:36.434623 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.434578 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:01:36.472236 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.472206 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b"] Nov 04 04:01:36.481519 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.481484 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh"] Nov 04 04:01:36.763005 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.762960 6000 generic.go:356] "Generic (PLEG): container finished" podID="52e771b7-2c73-4b2f-8266-f37cb72490bd" containerID="222676571a7b3bc0c14c3ee89cc468fc83b83d000a9a74d70b56d8567a73a7c4" exitCode=0 Nov 04 04:01:36.763489 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.763001 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s" event={"ID":"52e771b7-2c73-4b2f-8266-f37cb72490bd","Type":"ContainerDied","Data":"222676571a7b3bc0c14c3ee89cc468fc83b83d000a9a74d70b56d8567a73a7c4"} Nov 04 04:01:36.763699 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.763679 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s" event={"ID":"52e771b7-2c73-4b2f-8266-f37cb72490bd","Type":"ContainerDied","Data":"a6799e341eda5e4baab19d65c3a15af1e96c6cf7ae950e6cdb277a5d28f16e5c"} Nov 04 04:01:36.763831 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.763023 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s" Nov 04 04:01:36.764158 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.763708 6000 scope.go:122] "RemoveContainer" containerID="222676571a7b3bc0c14c3ee89cc468fc83b83d000a9a74d70b56d8567a73a7c4" Nov 04 04:01:36.768078 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.767978 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" event={"ID":"4afd26c6-87c7-4e7e-a315-5e4106ce8a21","Type":"ContainerStarted","Data":"a7bc59fec6b43e18b21b77cb26a6de51a2c010128600ece5a62aa2373eb0db96"} Nov 04 04:01:36.768078 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.768037 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" event={"ID":"4afd26c6-87c7-4e7e-a315-5e4106ce8a21","Type":"ContainerStarted","Data":"e092052b57012ed2d8d76062b4dc4a2a1156222a79b287536ce971025e851be7"} Nov 04 04:01:36.771304 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.771247 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" event={"ID":"78a5d7c4-f484-4788-a260-ead9c2ba30da","Type":"ContainerStarted","Data":"2e521d439765f43cc64e1d1576e3eeb17278594e5829e50a9f4865b57eff4de9"} Nov 04 04:01:36.771304 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.771299 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" event={"ID":"78a5d7c4-f484-4788-a260-ead9c2ba30da","Type":"ContainerStarted","Data":"edfdfc10a69c7c6e84faeccbc6323b5c15cab445449941b0a763ca32eddc190d"} Nov 04 04:01:36.778801 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.778748 6000 scope.go:122] "RemoveContainer" containerID="222676571a7b3bc0c14c3ee89cc468fc83b83d000a9a74d70b56d8567a73a7c4" Nov 04 04:01:36.782671 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:01:36.780198 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"222676571a7b3bc0c14c3ee89cc468fc83b83d000a9a74d70b56d8567a73a7c4\": container with ID starting with 222676571a7b3bc0c14c3ee89cc468fc83b83d000a9a74d70b56d8567a73a7c4 not found: ID does not exist" containerID="222676571a7b3bc0c14c3ee89cc468fc83b83d000a9a74d70b56d8567a73a7c4" Nov 04 04:01:36.782671 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.780233 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"222676571a7b3bc0c14c3ee89cc468fc83b83d000a9a74d70b56d8567a73a7c4"} err="failed to get container status \"222676571a7b3bc0c14c3ee89cc468fc83b83d000a9a74d70b56d8567a73a7c4\": rpc error: code = NotFound desc = could not find container \"222676571a7b3bc0c14c3ee89cc468fc83b83d000a9a74d70b56d8567a73a7c4\": container with ID starting with 222676571a7b3bc0c14c3ee89cc468fc83b83d000a9a74d70b56d8567a73a7c4 not found: ID does not exist" Nov 04 04:01:36.802285 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.798816 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" podStartSLOduration=1.798794955 podStartE2EDuration="1.798794955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:01:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:01:36.797081179 +0000 UTC m=+694.835521143" watchObservedRunningTime="2026-11-04 04:01:36.798794955 +0000 UTC m=+694.837234922" Nov 04 04:01:36.852385 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.852312 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s"] Nov 04 04:01:36.855712 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.855671 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-6a9cdb-6bd44cf46c-nwp8s"] Nov 04 04:01:36.864098 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.864031 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" podStartSLOduration=1.8640084799999999 podStartE2EDuration="1.86400848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:01:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:01:36.859558266 +0000 UTC m=+694.897998229" watchObservedRunningTime="2026-11-04 04:01:36.86400848 +0000 UTC m=+694.902448446" Nov 04 04:01:36.945159 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.944974 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:36.964469 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:36.964351 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l" podUID="31648502-f58d-43a9-a746-bcfb74eb70f7" containerName="payload-processing" containerID="cri-o://225646120e3d005645cd4800c05f154e4909d80f60de719df674448ad3021f91" gracePeriod=25 Nov 04 04:01:37.147922 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:37.147452 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:37.147922 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:37.147589 6000 patch_prober.go:29] interesting pod/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Nov 04 04:01:37.147922 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:37.147634 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" podUID="78a5d7c4-f484-4788-a260-ead9c2ba30da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Nov 04 04:01:37.948811 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:37.948774 6000 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Nov 04 04:01:37.949415 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:37.949382 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" podUID="4afd26c6-87c7-4e7e-a315-5e4106ce8a21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Nov 04 04:01:38.008810 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:38.008732 6000 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l" podUID="31648502-f58d-43a9-a746-bcfb74eb70f7" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 04 04:01:38.109673 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:38.105385 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="52e771b7-2c73-4b2f-8266-f37cb72490bd" path="/var/lib/kubelet/pods/52e771b7-2c73-4b2f-8266-f37cb72490bd/volumes" Nov 04 04:01:38.144554 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:38.144516 6000 patch_prober.go:29] interesting pod/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Nov 04 04:01:38.144842 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:38.144811 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" podUID="78a5d7c4-f484-4788-a260-ead9c2ba30da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Nov 04 04:01:38.235848 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:38.215382 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w"] Nov 04 04:01:38.235848 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:38.226857 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" podUID="5858d840-2308-4cf2-b7f3-9f73509bb196" containerName="istio-proxy" containerID="cri-o://4cd1001714fcfa5933cecb2b03775c1cd5dca827e8996ffaabfd0f66e68ad36f" gracePeriod=30 Nov 04 04:01:38.844227 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:38.844185 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg"] Nov 04 04:01:38.844747 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:38.844708 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" podUID="5869a8f0-b341-4616-b258-e5f6c9bab8f5" containerName="istio-proxy" containerID="cri-o://94926733e75992f8411eae66cdac09b45801589fb0ec3d065e4ee1cef2d254e8" gracePeriod=30 Nov 04 04:01:38.948238 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:38.948198 6000 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Nov 04 04:01:38.948497 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:38.948470 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" podUID="4afd26c6-87c7-4e7e-a315-5e4106ce8a21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Nov 04 04:01:39.138686 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:39.138598 6000 patch_prober.go:29] interesting pod/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Nov 04 04:01:39.139253 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:39.139219 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" podUID="78a5d7c4-f484-4788-a260-ead9c2ba30da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Nov 04 04:01:39.856572 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:39.856302 6000 generic.go:356] "Generic (PLEG): container finished" podID="48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d" containerID="c7d765357c5924ff5a248cb7e9f029accb6d15fc803813ec2214e341b8a636d8" exitCode=0 Nov 04 04:01:39.856572 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:39.856496 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" event={"ID":"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d","Type":"ContainerDied","Data":"c7d765357c5924ff5a248cb7e9f029accb6d15fc803813ec2214e341b8a636d8"} Nov 04 04:01:39.908654 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:39.908615 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r"] Nov 04 04:01:39.959623 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:39.959584 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd"] Nov 04 04:01:39.978316 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:39.961607 6000 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Nov 04 04:01:39.978316 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:39.977215 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" podUID="4afd26c6-87c7-4e7e-a315-5e4106ce8a21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Nov 04 04:01:39.979994 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:39.979942 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:01:40.088179 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.088140 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-credential-socket\") pod \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " Nov 04 04:01:40.088351 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.088228 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istiod-ca-cert\") pod \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " Nov 04 04:01:40.088351 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.088323 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-kube-api-access-rq6nt\" (UniqueName: \"kubernetes.io/projected/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-kube-api-access-rq6nt\") pod \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " Nov 04 04:01:40.088491 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.088369 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-workload-certs\") pod \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " Nov 04 04:01:40.088491 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.088400 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-podinfo\") pod \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " Nov 04 04:01:40.088605 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.088487 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-token\" (UniqueName: \"kubernetes.io/projected/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-token\") pod \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " Nov 04 04:01:40.088605 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.088517 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-data\") pod \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " Nov 04 04:01:40.088605 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.088598 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-workload-socket\") pod \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " Nov 04 04:01:40.088780 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.088641 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-envoy\") pod \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\" (UID: \"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d\") " Nov 04 04:01:40.091338 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.091295 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-envoy" pod "48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d" (UID: "48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:40.091523 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.091499 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-credential-socket" pod "48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d" (UID: "48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:40.091778 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.091736 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istiod-ca-cert" pod "48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d" (UID: "48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:01:40.092320 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.092289 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-workload-certs" pod "48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d" (UID: "48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:40.092531 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.092503 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-data" pod "48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d" (UID: "48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:40.092723 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.092681 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-workload-socket" pod "48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d" (UID: "48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:40.095207 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.095176 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-kube-api-access-rq6nt" pod "48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d" (UID: "48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d"). InnerVolumeSpecName "kube-api-access-rq6nt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:01:40.095469 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.095447 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-podinfo" pod "48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d" (UID: "48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 04:01:40.095994 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.095971 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-token" pod "48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d" (UID: "48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:01:40.139346 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.139248 6000 patch_prober.go:29] interesting pod/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Nov 04 04:01:40.142897 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.142850 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" podUID="78a5d7c4-f484-4788-a260-ead9c2ba30da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Nov 04 04:01:40.190731 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.190695 6000 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istiod-ca-cert\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:40.190731 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.190728 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rq6nt\" (UniqueName: \"kubernetes.io/projected/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-kube-api-access-rq6nt\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:40.190731 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.190739 6000 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-workload-certs\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:40.191028 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.190747 6000 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-podinfo\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:40.191028 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.190771 6000 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-token\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:40.191028 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.190781 6000 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-data\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:40.191028 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.190790 6000 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-workload-socket\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:40.191028 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.190798 6000 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-istio-envoy\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:40.191028 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.190807 6000 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d-credential-socket\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:40.873781 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.873508 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" event={"ID":"48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d","Type":"ContainerDied","Data":"0dad8552b1d01b128096674e5ceff65b89624a9867dfba7ca3ee30edd579df2d"} Nov 04 04:01:40.873781 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.873586 6000 scope.go:122] "RemoveContainer" containerID="c7d765357c5924ff5a248cb7e9f029accb6d15fc803813ec2214e341b8a636d8" Nov 04 04:01:40.873781 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.873734 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc" Nov 04 04:01:40.906615 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.906576 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc"] Nov 04 04:01:40.952783 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.950586 6000 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Nov 04 04:01:40.952783 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.950670 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" podUID="4afd26c6-87c7-4e7e-a315-5e4106ce8a21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Nov 04 04:01:40.972027 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:40.971989 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-6f2pc"] Nov 04 04:01:41.152841 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:41.152203 6000 patch_prober.go:29] interesting pod/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Nov 04 04:01:41.152841 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:41.152284 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" podUID="78a5d7c4-f484-4788-a260-ead9c2ba30da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Nov 04 04:01:41.165795 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:41.165095 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" podUID="6603cd37-3576-49ff-a87c-e307d4d35706" containerName="main" containerID="cri-o://179574c181fb521b043f92fb87e3834fd47ea2531a040f67e2f4c67d636e7abd" gracePeriod=45 Nov 04 04:01:41.264944 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:41.264903 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" podUID="f996ad18-5c32-4945-822e-62ed92214a54" containerName="main" containerID="cri-o://bb10a017c34e6d7847ea8accc778bae6f2832eadd493af8d23081e8d81bd6a0e" gracePeriod=45 Nov 04 04:01:41.282316 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:41.281977 6000 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" podUID="f996ad18-5c32-4945-822e-62ed92214a54" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": EOF" Nov 04 04:01:41.438559 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:41.438227 6000 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" podUID="6603cd37-3576-49ff-a87c-e307d4d35706" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": dial tcp 10.133.0.52:8000: connect: connection refused" Nov 04 04:01:41.947820 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:41.947202 6000 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Nov 04 04:01:41.947820 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:41.947280 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" podUID="4afd26c6-87c7-4e7e-a315-5e4106ce8a21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Nov 04 04:01:41.979795 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:41.965532 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 04:01:41.979795 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:41.967371 6000 generic.go:356] "Generic (PLEG): container finished" podID="6603cd37-3576-49ff-a87c-e307d4d35706" containerID="179574c181fb521b043f92fb87e3834fd47ea2531a040f67e2f4c67d636e7abd" exitCode=0 Nov 04 04:01:41.979795 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:41.967471 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" event={"ID":"6603cd37-3576-49ff-a87c-e307d4d35706","Type":"ContainerDied","Data":"179574c181fb521b043f92fb87e3834fd47ea2531a040f67e2f4c67d636e7abd"} Nov 04 04:01:41.979795 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:41.967511 6000 scope.go:122] "RemoveContainer" containerID="179574c181fb521b043f92fb87e3834fd47ea2531a040f67e2f4c67d636e7abd" Nov 04 04:01:41.985902 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:41.985732 6000 generic.go:356] "Generic (PLEG): container finished" podID="f996ad18-5c32-4945-822e-62ed92214a54" containerID="bb10a017c34e6d7847ea8accc778bae6f2832eadd493af8d23081e8d81bd6a0e" exitCode=0 Nov 04 04:01:41.986054 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:41.985821 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" event={"ID":"f996ad18-5c32-4945-822e-62ed92214a54","Type":"ContainerDied","Data":"bb10a017c34e6d7847ea8accc778bae6f2832eadd493af8d23081e8d81bd6a0e"} Nov 04 04:01:42.077596 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.073429 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6603cd37-3576-49ff-a87c-e307d4d35706-kube-api-access-hlqq7\" (UniqueName: \"kubernetes.io/projected/6603cd37-3576-49ff-a87c-e307d4d35706-kube-api-access-hlqq7\") pod \"6603cd37-3576-49ff-a87c-e307d4d35706\" (UID: \"6603cd37-3576-49ff-a87c-e307d4d35706\") " Nov 04 04:01:42.077596 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.074367 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-tmp-dir\") pod \"6603cd37-3576-49ff-a87c-e307d4d35706\" (UID: \"6603cd37-3576-49ff-a87c-e307d4d35706\") " Nov 04 04:01:42.077596 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.074457 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-dshm\") pod \"6603cd37-3576-49ff-a87c-e307d4d35706\" (UID: \"6603cd37-3576-49ff-a87c-e307d4d35706\") " Nov 04 04:01:42.077596 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.074541 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6603cd37-3576-49ff-a87c-e307d4d35706-tls-certs\" (UniqueName: \"kubernetes.io/secret/6603cd37-3576-49ff-a87c-e307d4d35706-tls-certs\") pod \"6603cd37-3576-49ff-a87c-e307d4d35706\" (UID: \"6603cd37-3576-49ff-a87c-e307d4d35706\") " Nov 04 04:01:42.092087 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.080195 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-tmp-dir" pod "6603cd37-3576-49ff-a87c-e307d4d35706" (UID: "6603cd37-3576-49ff-a87c-e307d4d35706"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:42.092087 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.082462 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-home\" (UniqueName: \"kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-home\") pod \"6603cd37-3576-49ff-a87c-e307d4d35706\" (UID: \"6603cd37-3576-49ff-a87c-e307d4d35706\") " Nov 04 04:01:42.092087 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.082539 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-model-cache\") pod \"6603cd37-3576-49ff-a87c-e307d4d35706\" (UID: \"6603cd37-3576-49ff-a87c-e307d4d35706\") " Nov 04 04:01:42.092087 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.082739 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-home" pod "6603cd37-3576-49ff-a87c-e307d4d35706" (UID: "6603cd37-3576-49ff-a87c-e307d4d35706"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:42.092087 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.082839 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6603cd37-3576-49ff-a87c-e307d4d35706-kube-api-access-hlqq7" pod "6603cd37-3576-49ff-a87c-e307d4d35706" (UID: "6603cd37-3576-49ff-a87c-e307d4d35706"). InnerVolumeSpecName "kube-api-access-hlqq7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:01:42.092087 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.083064 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-model-cache" pod "6603cd37-3576-49ff-a87c-e307d4d35706" (UID: "6603cd37-3576-49ff-a87c-e307d4d35706"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:42.092087 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.083208 6000 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-tmp-dir\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:42.092087 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.083227 6000 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-home\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:42.092087 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.083241 6000 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-model-cache\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:42.092087 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.083257 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlqq7\" (UniqueName: \"kubernetes.io/projected/6603cd37-3576-49ff-a87c-e307d4d35706-kube-api-access-hlqq7\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:42.092087 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.084977 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-dshm" pod "6603cd37-3576-49ff-a87c-e307d4d35706" (UID: "6603cd37-3576-49ff-a87c-e307d4d35706"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:42.092087 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.086489 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6603cd37-3576-49ff-a87c-e307d4d35706-tls-certs" pod "6603cd37-3576-49ff-a87c-e307d4d35706" (UID: "6603cd37-3576-49ff-a87c-e307d4d35706"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 04:01:42.092087 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.088625 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d" path="/var/lib/kubelet/pods/48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d/volumes" Nov 04 04:01:42.164796 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.141829 6000 patch_prober.go:29] interesting pod/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Nov 04 04:01:42.164796 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.141910 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" podUID="78a5d7c4-f484-4788-a260-ead9c2ba30da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Nov 04 04:01:42.224785 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.199913 6000 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6603cd37-3576-49ff-a87c-e307d4d35706-tls-certs\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:42.224785 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.199953 6000 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6603cd37-3576-49ff-a87c-e307d4d35706-dshm\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:42.289773 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.289732 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 04:01:42.402606 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.402499 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f996ad18-5c32-4945-822e-62ed92214a54-kube-api-access-lvv2h\" (UniqueName: \"kubernetes.io/projected/f996ad18-5c32-4945-822e-62ed92214a54-kube-api-access-lvv2h\") pod \"f996ad18-5c32-4945-822e-62ed92214a54\" (UID: \"f996ad18-5c32-4945-822e-62ed92214a54\") " Nov 04 04:01:42.402924 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.402900 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f996ad18-5c32-4945-822e-62ed92214a54-tls-certs\" (UniqueName: \"kubernetes.io/secret/f996ad18-5c32-4945-822e-62ed92214a54-tls-certs\") pod \"f996ad18-5c32-4945-822e-62ed92214a54\" (UID: \"f996ad18-5c32-4945-822e-62ed92214a54\") " Nov 04 04:01:42.403136 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.403120 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-home\" (UniqueName: \"kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-home\") pod \"f996ad18-5c32-4945-822e-62ed92214a54\" (UID: \"f996ad18-5c32-4945-822e-62ed92214a54\") " Nov 04 04:01:42.403390 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.403363 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-home" pod "f996ad18-5c32-4945-822e-62ed92214a54" (UID: "f996ad18-5c32-4945-822e-62ed92214a54"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:42.403522 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.403503 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-model-cache\") pod \"f996ad18-5c32-4945-822e-62ed92214a54\" (UID: \"f996ad18-5c32-4945-822e-62ed92214a54\") " Nov 04 04:01:42.403706 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.403670 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-tmp-dir\") pod \"f996ad18-5c32-4945-822e-62ed92214a54\" (UID: \"f996ad18-5c32-4945-822e-62ed92214a54\") " Nov 04 04:01:42.403959 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.403911 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-tmp-dir" pod "f996ad18-5c32-4945-822e-62ed92214a54" (UID: "f996ad18-5c32-4945-822e-62ed92214a54"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:42.404334 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.403947 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-dshm\") pod \"f996ad18-5c32-4945-822e-62ed92214a54\" (UID: \"f996ad18-5c32-4945-822e-62ed92214a54\") " Nov 04 04:01:42.404452 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.404430 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-model-cache" pod "f996ad18-5c32-4945-822e-62ed92214a54" (UID: "f996ad18-5c32-4945-822e-62ed92214a54"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:42.404548 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.404468 6000 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-home\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:42.404650 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.404637 6000 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-tmp-dir\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:42.405463 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.405433 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f996ad18-5c32-4945-822e-62ed92214a54-kube-api-access-lvv2h" pod "f996ad18-5c32-4945-822e-62ed92214a54" (UID: "f996ad18-5c32-4945-822e-62ed92214a54"). InnerVolumeSpecName "kube-api-access-lvv2h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:01:42.405851 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.405824 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f996ad18-5c32-4945-822e-62ed92214a54-tls-certs" pod "f996ad18-5c32-4945-822e-62ed92214a54" (UID: "f996ad18-5c32-4945-822e-62ed92214a54"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 04:01:42.406268 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.406244 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-dshm" pod "f996ad18-5c32-4945-822e-62ed92214a54" (UID: "f996ad18-5c32-4945-822e-62ed92214a54"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:42.505963 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.505925 6000 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-model-cache\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:42.505963 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.505961 6000 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f996ad18-5c32-4945-822e-62ed92214a54-dshm\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:42.506175 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.505978 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvv2h\" (UniqueName: \"kubernetes.io/projected/f996ad18-5c32-4945-822e-62ed92214a54-kube-api-access-lvv2h\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:42.506175 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.505993 6000 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f996ad18-5c32-4945-822e-62ed92214a54-tls-certs\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:42.947316 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.947274 6000 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Nov 04 04:01:42.947505 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.947341 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" podUID="4afd26c6-87c7-4e7e-a315-5e4106ce8a21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Nov 04 04:01:43.001016 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.998486 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" Nov 04 04:01:43.001016 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:42.998954 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg" event={"ID":"6603cd37-3576-49ff-a87c-e307d4d35706","Type":"ContainerDied","Data":"59e0c7c0a746e65776b9f12edd3fd33fdb4f549c5bc8b602c6f6d7b70b984341"} Nov 04 04:01:43.001947 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.001730 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" event={"ID":"f996ad18-5c32-4945-822e-62ed92214a54","Type":"ContainerDied","Data":"2cb80b764341d73ca3f1b34319f318e4fd493a5a307bd9669f061423a983e6c4"} Nov 04 04:01:43.001947 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.001790 6000 scope.go:122] "RemoveContainer" containerID="bb10a017c34e6d7847ea8accc778bae6f2832eadd493af8d23081e8d81bd6a0e" Nov 04 04:01:43.001947 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.001861 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" Nov 04 04:01:43.039614 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.039576 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg"] Nov 04 04:01:43.044303 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.043768 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-56a44f/premium-w5-56a44f-kserve-b7d69c94f-987lg"] Nov 04 04:01:43.071468 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.071376 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr"] Nov 04 04:01:43.082108 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.082076 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr"] Nov 04 04:01:43.142195 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.142148 6000 patch_prober.go:29] interesting pod/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Nov 04 04:01:43.142409 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.142217 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" podUID="78a5d7c4-f484-4788-a260-ead9c2ba30da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Nov 04 04:01:43.198848 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.198807 6000 patch_prober.go:29] interesting pod/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 04:01:43.199309 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.198870 6000 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" podUID="5869a8f0-b341-4616-b258-e5f6c9bab8f5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 04:01:43.279334 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.278826 6000 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-56a44f/simulator-w5-56a44f-kserve-779787b884-jhjdr" podUID="f996ad18-5c32-4945-822e-62ed92214a54" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 04 04:01:43.420656 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.420568 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w"] Nov 04 04:01:43.421593 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.421561 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" podUID="599a125f-8022-4f5a-9384-e54dd8e13b32" containerName="istio-proxy" containerID="cri-o://75507b9736352520d19b589257ee1dd35be205c5986d354d8df2188362afd4b2" gracePeriod=30 Nov 04 04:01:43.770342 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.770313 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 04:01:43.829813 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.829774 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5858d840-2308-4cf2-b7f3-9f73509bb196-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5858d840-2308-4cf2-b7f3-9f73509bb196-istiod-ca-cert\") pod \"5858d840-2308-4cf2-b7f3-9f73509bb196\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " Nov 04 04:01:43.830031 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.829905 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-workload-socket\") pod \"5858d840-2308-4cf2-b7f3-9f73509bb196\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " Nov 04 04:01:43.830031 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.829941 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-data\") pod \"5858d840-2308-4cf2-b7f3-9f73509bb196\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " Nov 04 04:01:43.830031 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.829980 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-envoy\") pod \"5858d840-2308-4cf2-b7f3-9f73509bb196\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " Nov 04 04:01:43.830031 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.830016 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-workload-certs\") pod \"5858d840-2308-4cf2-b7f3-9f73509bb196\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " Nov 04 04:01:43.830266 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.830084 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-podinfo\") pod \"5858d840-2308-4cf2-b7f3-9f73509bb196\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " Nov 04 04:01:43.830266 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.830142 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5858d840-2308-4cf2-b7f3-9f73509bb196-kube-api-access-nlc4r\" (UniqueName: \"kubernetes.io/projected/5858d840-2308-4cf2-b7f3-9f73509bb196-kube-api-access-nlc4r\") pod \"5858d840-2308-4cf2-b7f3-9f73509bb196\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " Nov 04 04:01:43.830266 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.830210 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-credential-socket\") pod \"5858d840-2308-4cf2-b7f3-9f73509bb196\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " Nov 04 04:01:43.830266 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.830261 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-token\" (UniqueName: \"kubernetes.io/projected/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-token\") pod \"5858d840-2308-4cf2-b7f3-9f73509bb196\" (UID: \"5858d840-2308-4cf2-b7f3-9f73509bb196\") " Nov 04 04:01:43.831904 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.831343 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-workload-socket" pod "5858d840-2308-4cf2-b7f3-9f73509bb196" (UID: "5858d840-2308-4cf2-b7f3-9f73509bb196"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:43.831904 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.831518 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-credential-socket" pod "5858d840-2308-4cf2-b7f3-9f73509bb196" (UID: "5858d840-2308-4cf2-b7f3-9f73509bb196"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:43.831904 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.831670 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5858d840-2308-4cf2-b7f3-9f73509bb196-istiod-ca-cert" pod "5858d840-2308-4cf2-b7f3-9f73509bb196" (UID: "5858d840-2308-4cf2-b7f3-9f73509bb196"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:01:43.832537 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.832457 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-workload-certs" pod "5858d840-2308-4cf2-b7f3-9f73509bb196" (UID: "5858d840-2308-4cf2-b7f3-9f73509bb196"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:43.832537 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.832527 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-data" pod "5858d840-2308-4cf2-b7f3-9f73509bb196" (UID: "5858d840-2308-4cf2-b7f3-9f73509bb196"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:43.835903 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.835858 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5858d840-2308-4cf2-b7f3-9f73509bb196-kube-api-access-nlc4r" pod "5858d840-2308-4cf2-b7f3-9f73509bb196" (UID: "5858d840-2308-4cf2-b7f3-9f73509bb196"). InnerVolumeSpecName "kube-api-access-nlc4r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:01:43.836121 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.836038 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-token" pod "5858d840-2308-4cf2-b7f3-9f73509bb196" (UID: "5858d840-2308-4cf2-b7f3-9f73509bb196"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:01:43.836121 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.836086 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-envoy" pod "5858d840-2308-4cf2-b7f3-9f73509bb196" (UID: "5858d840-2308-4cf2-b7f3-9f73509bb196"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:43.837195 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.837166 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-podinfo" pod "5858d840-2308-4cf2-b7f3-9f73509bb196" (UID: "5858d840-2308-4cf2-b7f3-9f73509bb196"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 04:01:43.932015 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.931838 6000 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-workload-socket\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:43.932015 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.931877 6000 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-data\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:43.932015 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.931892 6000 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-envoy\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:43.932015 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.931905 6000 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-workload-certs\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:43.932015 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.931918 6000 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-podinfo\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:43.932015 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.931932 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlc4r\" (UniqueName: \"kubernetes.io/projected/5858d840-2308-4cf2-b7f3-9f73509bb196-kube-api-access-nlc4r\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:43.932015 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.931945 6000 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5858d840-2308-4cf2-b7f3-9f73509bb196-credential-socket\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:43.932015 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.931957 6000 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5858d840-2308-4cf2-b7f3-9f73509bb196-istio-token\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:43.932015 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:43.931969 6000 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5858d840-2308-4cf2-b7f3-9f73509bb196-istiod-ca-cert\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:44.030036 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.028606 6000 generic.go:356] "Generic (PLEG): container finished" podID="5869a8f0-b341-4616-b258-e5f6c9bab8f5" containerID="94926733e75992f8411eae66cdac09b45801589fb0ec3d065e4ee1cef2d254e8" exitCode=0 Nov 04 04:01:44.030036 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.028876 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" event={"ID":"5869a8f0-b341-4616-b258-e5f6c9bab8f5","Type":"ContainerDied","Data":"94926733e75992f8411eae66cdac09b45801589fb0ec3d065e4ee1cef2d254e8"} Nov 04 04:01:44.034512 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.034472 6000 generic.go:356] "Generic (PLEG): container finished" podID="5858d840-2308-4cf2-b7f3-9f73509bb196" containerID="4cd1001714fcfa5933cecb2b03775c1cd5dca827e8996ffaabfd0f66e68ad36f" exitCode=0 Nov 04 04:01:44.034693 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.034537 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" event={"ID":"5858d840-2308-4cf2-b7f3-9f73509bb196","Type":"ContainerDied","Data":"4cd1001714fcfa5933cecb2b03775c1cd5dca827e8996ffaabfd0f66e68ad36f"} Nov 04 04:01:44.034693 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.034590 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" event={"ID":"5858d840-2308-4cf2-b7f3-9f73509bb196","Type":"ContainerDied","Data":"4fc3c2eb3f2c1d65d957904aa75a50b194fc929317d96e40755089603a494ea6"} Nov 04 04:01:44.034693 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.034622 6000 scope.go:122] "RemoveContainer" containerID="4cd1001714fcfa5933cecb2b03775c1cd5dca827e8996ffaabfd0f66e68ad36f" Nov 04 04:01:44.034907 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.034772 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w" Nov 04 04:01:44.088279 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.088248 6000 scope.go:122] "RemoveContainer" containerID="0701a9639a9b39871739c0fc3256eef7bec20e71d2fcc43068b235bfab0ddece" Nov 04 04:01:44.094890 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.094633 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6603cd37-3576-49ff-a87c-e307d4d35706" path="/var/lib/kubelet/pods/6603cd37-3576-49ff-a87c-e307d4d35706/volumes" Nov 04 04:01:44.096808 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.095665 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f996ad18-5c32-4945-822e-62ed92214a54" path="/var/lib/kubelet/pods/f996ad18-5c32-4945-822e-62ed92214a54/volumes" Nov 04 04:01:44.107593 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.107355 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w"] Nov 04 04:01:44.113806 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.108463 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-clh9w"] Nov 04 04:01:44.199310 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.199276 6000 scope.go:122] "RemoveContainer" containerID="4cd1001714fcfa5933cecb2b03775c1cd5dca827e8996ffaabfd0f66e68ad36f" Nov 04 04:01:44.200433 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:01:44.200387 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cd1001714fcfa5933cecb2b03775c1cd5dca827e8996ffaabfd0f66e68ad36f\": container with ID starting with 4cd1001714fcfa5933cecb2b03775c1cd5dca827e8996ffaabfd0f66e68ad36f not found: ID does not exist" containerID="4cd1001714fcfa5933cecb2b03775c1cd5dca827e8996ffaabfd0f66e68ad36f" Nov 04 04:01:44.200746 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.200629 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cd1001714fcfa5933cecb2b03775c1cd5dca827e8996ffaabfd0f66e68ad36f"} err="failed to get container status \"4cd1001714fcfa5933cecb2b03775c1cd5dca827e8996ffaabfd0f66e68ad36f\": rpc error: code = NotFound desc = could not find container \"4cd1001714fcfa5933cecb2b03775c1cd5dca827e8996ffaabfd0f66e68ad36f\": container with ID starting with 4cd1001714fcfa5933cecb2b03775c1cd5dca827e8996ffaabfd0f66e68ad36f not found: ID does not exist" Nov 04 04:01:44.200746 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.200685 6000 scope.go:122] "RemoveContainer" containerID="0701a9639a9b39871739c0fc3256eef7bec20e71d2fcc43068b235bfab0ddece" Nov 04 04:01:44.216044 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:01:44.215902 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0701a9639a9b39871739c0fc3256eef7bec20e71d2fcc43068b235bfab0ddece\": container with ID starting with 0701a9639a9b39871739c0fc3256eef7bec20e71d2fcc43068b235bfab0ddece not found: ID does not exist" containerID="0701a9639a9b39871739c0fc3256eef7bec20e71d2fcc43068b235bfab0ddece" Nov 04 04:01:44.216044 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.215952 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0701a9639a9b39871739c0fc3256eef7bec20e71d2fcc43068b235bfab0ddece"} err="failed to get container status \"0701a9639a9b39871739c0fc3256eef7bec20e71d2fcc43068b235bfab0ddece\": rpc error: code = NotFound desc = could not find container \"0701a9639a9b39871739c0fc3256eef7bec20e71d2fcc43068b235bfab0ddece\": container with ID starting with 0701a9639a9b39871739c0fc3256eef7bec20e71d2fcc43068b235bfab0ddece not found: ID does not exist" Nov 04 04:01:44.478712 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.478673 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 04:01:44.562785 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.562657 6000 patch_prober.go:29] interesting pod/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 04:01:44.563094 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.562749 6000 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" podUID="599a125f-8022-4f5a-9384-e54dd8e13b32" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 04:01:44.568786 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.567724 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-workload-socket\") pod \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " Nov 04 04:01:44.568786 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.567816 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-credential-socket\") pod \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " Nov 04 04:01:44.568786 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.567855 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-envoy\") pod \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " Nov 04 04:01:44.568786 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.567960 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-data\") pod \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " Nov 04 04:01:44.568786 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.568001 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istiod-ca-cert\") pod \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " Nov 04 04:01:44.568786 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.568040 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-workload-certs\") pod \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " Nov 04 04:01:44.568786 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.568073 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-podinfo\") pod \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " Nov 04 04:01:44.568786 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.568117 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5869a8f0-b341-4616-b258-e5f6c9bab8f5-kube-api-access-nh4ts\" (UniqueName: \"kubernetes.io/projected/5869a8f0-b341-4616-b258-e5f6c9bab8f5-kube-api-access-nh4ts\") pod \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " Nov 04 04:01:44.568786 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.568149 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-token\" (UniqueName: \"kubernetes.io/projected/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-token\") pod \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\" (UID: \"5869a8f0-b341-4616-b258-e5f6c9bab8f5\") " Nov 04 04:01:44.570099 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.570073 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-data" pod "5869a8f0-b341-4616-b258-e5f6c9bab8f5" (UID: "5869a8f0-b341-4616-b258-e5f6c9bab8f5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:44.570459 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.570416 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-workload-socket" pod "5869a8f0-b341-4616-b258-e5f6c9bab8f5" (UID: "5869a8f0-b341-4616-b258-e5f6c9bab8f5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:44.570789 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.570768 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-credential-socket" pod "5869a8f0-b341-4616-b258-e5f6c9bab8f5" (UID: "5869a8f0-b341-4616-b258-e5f6c9bab8f5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:44.573935 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.573898 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istiod-ca-cert" pod "5869a8f0-b341-4616-b258-e5f6c9bab8f5" (UID: "5869a8f0-b341-4616-b258-e5f6c9bab8f5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:01:44.578171 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.578143 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-workload-certs" pod "5869a8f0-b341-4616-b258-e5f6c9bab8f5" (UID: "5869a8f0-b341-4616-b258-e5f6c9bab8f5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:44.580067 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.580041 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-envoy" pod "5869a8f0-b341-4616-b258-e5f6c9bab8f5" (UID: "5869a8f0-b341-4616-b258-e5f6c9bab8f5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:44.582579 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.582539 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-podinfo" pod "5869a8f0-b341-4616-b258-e5f6c9bab8f5" (UID: "5869a8f0-b341-4616-b258-e5f6c9bab8f5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 04:01:44.583137 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.583099 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5869a8f0-b341-4616-b258-e5f6c9bab8f5-kube-api-access-nh4ts" pod "5869a8f0-b341-4616-b258-e5f6c9bab8f5" (UID: "5869a8f0-b341-4616-b258-e5f6c9bab8f5"). InnerVolumeSpecName "kube-api-access-nh4ts". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:01:44.585659 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.585614 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-token" pod "5869a8f0-b341-4616-b258-e5f6c9bab8f5" (UID: "5869a8f0-b341-4616-b258-e5f6c9bab8f5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:01:44.669826 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.669783 6000 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-data\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:44.669826 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.669831 6000 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istiod-ca-cert\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:44.670105 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.669853 6000 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-workload-certs\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:44.670105 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.669870 6000 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-podinfo\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:44.670105 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.669887 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nh4ts\" (UniqueName: \"kubernetes.io/projected/5869a8f0-b341-4616-b258-e5f6c9bab8f5-kube-api-access-nh4ts\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:44.670105 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.669902 6000 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-token\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:44.670105 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.669915 6000 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-workload-socket\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:44.670105 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.669929 6000 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-credential-socket\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:44.670105 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.669943 6000 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5869a8f0-b341-4616-b258-e5f6c9bab8f5-istio-envoy\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:44.725304 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.724416 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:44.725865 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.725776 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:44.726926 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.726904 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:01:44.910024 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.909277 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r" podUID="e48d6395-918f-4ebd-91c9-4b93edbb520d" containerName="payload-processing" containerID="cri-o://372e9783aff79dc67f2838732ec1a84b0c5a0509e771c12e06aca755e011446e" gracePeriod=25 Nov 04 04:01:44.953306 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.953263 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:44.954257 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.954221 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:44.955119 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.955098 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:01:44.978133 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:44.978008 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd" podUID="200578c9-4fac-445d-b697-55d8d79310fb" containerName="payload-pre-processing" containerID="cri-o://0064031ada67f2e95496b8d794f38f84a17c5064ce56c02fa44a64d5025fb1c3" gracePeriod=25 Nov 04 04:01:45.057491 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:45.057447 6000 generic.go:356] "Generic (PLEG): container finished" podID="e48d6395-918f-4ebd-91c9-4b93edbb520d" containerID="372e9783aff79dc67f2838732ec1a84b0c5a0509e771c12e06aca755e011446e" exitCode=0 Nov 04 04:01:45.057734 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:45.057580 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r" event={"ID":"e48d6395-918f-4ebd-91c9-4b93edbb520d","Type":"ContainerDied","Data":"372e9783aff79dc67f2838732ec1a84b0c5a0509e771c12e06aca755e011446e"} Nov 04 04:01:45.061817 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:45.061767 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" Nov 04 04:01:45.062853 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:45.062121 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg" event={"ID":"5869a8f0-b341-4616-b258-e5f6c9bab8f5","Type":"ContainerDied","Data":"e1463ce35807dc65cb447caca803fef6e6a7490c9d9b9592edb7493765234310"} Nov 04 04:01:45.062853 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:45.062189 6000 scope.go:122] "RemoveContainer" containerID="94926733e75992f8411eae66cdac09b45801589fb0ec3d065e4ee1cef2d254e8" Nov 04 04:01:45.116360 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:45.116318 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg"] Nov 04 04:01:45.120339 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:45.120308 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-hjhdg"] Nov 04 04:01:45.323145 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:45.322828 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r" Nov 04 04:01:45.393626 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:45.393589 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd" Nov 04 04:01:45.396449 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:45.396413 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e48d6395-918f-4ebd-91c9-4b93edbb520d-kube-api-access-gv8lp\" (UniqueName: \"kubernetes.io/projected/e48d6395-918f-4ebd-91c9-4b93edbb520d-kube-api-access-gv8lp\") pod \"e48d6395-918f-4ebd-91c9-4b93edbb520d\" (UID: \"e48d6395-918f-4ebd-91c9-4b93edbb520d\") " Nov 04 04:01:45.396589 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:45.396536 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e48d6395-918f-4ebd-91c9-4b93edbb520d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e48d6395-918f-4ebd-91c9-4b93edbb520d-plugins-config-volume\") pod \"e48d6395-918f-4ebd-91c9-4b93edbb520d\" (UID: \"e48d6395-918f-4ebd-91c9-4b93edbb520d\") " Nov 04 04:01:45.397261 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:45.397205 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e48d6395-918f-4ebd-91c9-4b93edbb520d-plugins-config-volume" pod "e48d6395-918f-4ebd-91c9-4b93edbb520d" (UID: "e48d6395-918f-4ebd-91c9-4b93edbb520d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:01:45.399250 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:45.399186 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e48d6395-918f-4ebd-91c9-4b93edbb520d-kube-api-access-gv8lp" pod "e48d6395-918f-4ebd-91c9-4b93edbb520d" (UID: "e48d6395-918f-4ebd-91c9-4b93edbb520d"). InnerVolumeSpecName "kube-api-access-gv8lp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:01:45.498237 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:45.498178 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/200578c9-4fac-445d-b697-55d8d79310fb-kube-api-access-zs89s\" (UniqueName: \"kubernetes.io/projected/200578c9-4fac-445d-b697-55d8d79310fb-kube-api-access-zs89s\") pod \"200578c9-4fac-445d-b697-55d8d79310fb\" (UID: \"200578c9-4fac-445d-b697-55d8d79310fb\") " Nov 04 04:01:45.498557 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:45.498338 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/200578c9-4fac-445d-b697-55d8d79310fb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/200578c9-4fac-445d-b697-55d8d79310fb-plugins-config-volume\") pod \"200578c9-4fac-445d-b697-55d8d79310fb\" (UID: \"200578c9-4fac-445d-b697-55d8d79310fb\") " Nov 04 04:01:45.498671 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:45.498659 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gv8lp\" (UniqueName: \"kubernetes.io/projected/e48d6395-918f-4ebd-91c9-4b93edbb520d-kube-api-access-gv8lp\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:45.498790 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:45.498679 6000 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e48d6395-918f-4ebd-91c9-4b93edbb520d-plugins-config-volume\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:45.499058 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:45.499034 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/200578c9-4fac-445d-b697-55d8d79310fb-plugins-config-volume" pod "200578c9-4fac-445d-b697-55d8d79310fb" (UID: "200578c9-4fac-445d-b697-55d8d79310fb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:01:45.500769 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:45.500734 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/200578c9-4fac-445d-b697-55d8d79310fb-kube-api-access-zs89s" pod "200578c9-4fac-445d-b697-55d8d79310fb" (UID: "200578c9-4fac-445d-b697-55d8d79310fb"). InnerVolumeSpecName "kube-api-access-zs89s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:01:45.599905 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:45.599866 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zs89s\" (UniqueName: \"kubernetes.io/projected/200578c9-4fac-445d-b697-55d8d79310fb-kube-api-access-zs89s\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:45.599905 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:45.599898 6000 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/200578c9-4fac-445d-b697-55d8d79310fb-plugins-config-volume\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:46.082063 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:46.082023 6000 generic.go:356] "Generic (PLEG): container finished" podID="200578c9-4fac-445d-b697-55d8d79310fb" containerID="0064031ada67f2e95496b8d794f38f84a17c5064ce56c02fa44a64d5025fb1c3" exitCode=0 Nov 04 04:01:46.082245 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:46.082132 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd" Nov 04 04:01:46.085243 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:46.085215 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r" Nov 04 04:01:46.086559 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:46.086535 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5858d840-2308-4cf2-b7f3-9f73509bb196" path="/var/lib/kubelet/pods/5858d840-2308-4cf2-b7f3-9f73509bb196/volumes" Nov 04 04:01:46.087412 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:46.087387 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5869a8f0-b341-4616-b258-e5f6c9bab8f5" path="/var/lib/kubelet/pods/5869a8f0-b341-4616-b258-e5f6c9bab8f5/volumes" Nov 04 04:01:46.088116 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:46.088092 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd" event={"ID":"200578c9-4fac-445d-b697-55d8d79310fb","Type":"ContainerDied","Data":"0064031ada67f2e95496b8d794f38f84a17c5064ce56c02fa44a64d5025fb1c3"} Nov 04 04:01:46.088222 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:46.088153 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd" event={"ID":"200578c9-4fac-445d-b697-55d8d79310fb","Type":"ContainerDied","Data":"259867f16c126adcc0441ca1307b4b9150dadb668f2523667116436fd14074c0"} Nov 04 04:01:46.088222 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:46.088172 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r" event={"ID":"e48d6395-918f-4ebd-91c9-4b93edbb520d","Type":"ContainerDied","Data":"e1323e2ed1718f262e315bc13d1f8faa32e0c1a24a0a11f0a73360117effc811"} Nov 04 04:01:46.088222 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:46.088196 6000 scope.go:122] "RemoveContainer" containerID="0064031ada67f2e95496b8d794f38f84a17c5064ce56c02fa44a64d5025fb1c3" Nov 04 04:01:46.120435 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:46.120362 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd"] Nov 04 04:01:46.129326 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:46.129281 6000 scope.go:122] "RemoveContainer" containerID="0064031ada67f2e95496b8d794f38f84a17c5064ce56c02fa44a64d5025fb1c3" Nov 04 04:01:46.151674 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:01:46.151555 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0064031ada67f2e95496b8d794f38f84a17c5064ce56c02fa44a64d5025fb1c3\": container with ID starting with 0064031ada67f2e95496b8d794f38f84a17c5064ce56c02fa44a64d5025fb1c3 not found: ID does not exist" containerID="0064031ada67f2e95496b8d794f38f84a17c5064ce56c02fa44a64d5025fb1c3" Nov 04 04:01:46.151674 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:46.151626 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0064031ada67f2e95496b8d794f38f84a17c5064ce56c02fa44a64d5025fb1c3"} err="failed to get container status \"0064031ada67f2e95496b8d794f38f84a17c5064ce56c02fa44a64d5025fb1c3\": rpc error: code = NotFound desc = could not find container \"0064031ada67f2e95496b8d794f38f84a17c5064ce56c02fa44a64d5025fb1c3\": container with ID starting with 0064031ada67f2e95496b8d794f38f84a17c5064ce56c02fa44a64d5025fb1c3 not found: ID does not exist" Nov 04 04:01:46.152068 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:46.152020 6000 scope.go:122] "RemoveContainer" containerID="372e9783aff79dc67f2838732ec1a84b0c5a0509e771c12e06aca755e011446e" Nov 04 04:01:46.162318 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:46.162276 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-64c20d-5c65bd88c5-hrbnd"] Nov 04 04:01:46.189734 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:46.189662 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r"] Nov 04 04:01:46.193390 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:46.193356 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-64c20d-574b8745d4-m8m7r"] Nov 04 04:01:48.007348 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.007312 6000 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l" podUID="31648502-f58d-43a9-a746-bcfb74eb70f7" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 04 04:01:48.087393 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.087333 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="200578c9-4fac-445d-b697-55d8d79310fb" path="/var/lib/kubelet/pods/200578c9-4fac-445d-b697-55d8d79310fb/volumes" Nov 04 04:01:48.087990 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.087967 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e48d6395-918f-4ebd-91c9-4b93edbb520d" path="/var/lib/kubelet/pods/e48d6395-918f-4ebd-91c9-4b93edbb520d/volumes" Nov 04 04:01:48.634437 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.634409 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:01:48.738995 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.738896 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/599a125f-8022-4f5a-9384-e54dd8e13b32-kube-api-access-m5x69\" (UniqueName: \"kubernetes.io/projected/599a125f-8022-4f5a-9384-e54dd8e13b32-kube-api-access-m5x69\") pod \"599a125f-8022-4f5a-9384-e54dd8e13b32\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " Nov 04 04:01:48.738995 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.738958 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-podinfo\") pod \"599a125f-8022-4f5a-9384-e54dd8e13b32\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " Nov 04 04:01:48.739240 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.739000 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-token\" (UniqueName: \"kubernetes.io/projected/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-token\") pod \"599a125f-8022-4f5a-9384-e54dd8e13b32\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " Nov 04 04:01:48.739240 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.739050 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-data\") pod \"599a125f-8022-4f5a-9384-e54dd8e13b32\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " Nov 04 04:01:48.739240 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.739085 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-envoy\") pod \"599a125f-8022-4f5a-9384-e54dd8e13b32\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " Nov 04 04:01:48.739240 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.739121 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/599a125f-8022-4f5a-9384-e54dd8e13b32-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/599a125f-8022-4f5a-9384-e54dd8e13b32-istiod-ca-cert\") pod \"599a125f-8022-4f5a-9384-e54dd8e13b32\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " Nov 04 04:01:48.739240 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.739168 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-workload-socket\") pod \"599a125f-8022-4f5a-9384-e54dd8e13b32\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " Nov 04 04:01:48.739240 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.739212 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-credential-socket\") pod \"599a125f-8022-4f5a-9384-e54dd8e13b32\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " Nov 04 04:01:48.739562 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.739368 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-workload-certs\") pod \"599a125f-8022-4f5a-9384-e54dd8e13b32\" (UID: \"599a125f-8022-4f5a-9384-e54dd8e13b32\") " Nov 04 04:01:48.739562 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.739438 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-data" pod "599a125f-8022-4f5a-9384-e54dd8e13b32" (UID: "599a125f-8022-4f5a-9384-e54dd8e13b32"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:48.739562 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.739539 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/599a125f-8022-4f5a-9384-e54dd8e13b32-istiod-ca-cert" pod "599a125f-8022-4f5a-9384-e54dd8e13b32" (UID: "599a125f-8022-4f5a-9384-e54dd8e13b32"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:01:48.739709 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.739644 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-workload-socket" pod "599a125f-8022-4f5a-9384-e54dd8e13b32" (UID: "599a125f-8022-4f5a-9384-e54dd8e13b32"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:48.739863 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.739823 6000 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-workload-socket\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:48.739863 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.739831 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-credential-socket" pod "599a125f-8022-4f5a-9384-e54dd8e13b32" (UID: "599a125f-8022-4f5a-9384-e54dd8e13b32"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:48.740050 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.739847 6000 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-data\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:48.740050 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.739898 6000 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/599a125f-8022-4f5a-9384-e54dd8e13b32-istiod-ca-cert\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:48.740050 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.739901 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-workload-certs" pod "599a125f-8022-4f5a-9384-e54dd8e13b32" (UID: "599a125f-8022-4f5a-9384-e54dd8e13b32"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:48.741902 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.741870 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-token" pod "599a125f-8022-4f5a-9384-e54dd8e13b32" (UID: "599a125f-8022-4f5a-9384-e54dd8e13b32"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:01:48.742001 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.741895 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/599a125f-8022-4f5a-9384-e54dd8e13b32-kube-api-access-m5x69" pod "599a125f-8022-4f5a-9384-e54dd8e13b32" (UID: "599a125f-8022-4f5a-9384-e54dd8e13b32"). InnerVolumeSpecName "kube-api-access-m5x69". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:01:48.742001 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.741976 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-podinfo" pod "599a125f-8022-4f5a-9384-e54dd8e13b32" (UID: "599a125f-8022-4f5a-9384-e54dd8e13b32"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 04:01:48.742361 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.742330 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-envoy" pod "599a125f-8022-4f5a-9384-e54dd8e13b32" (UID: "599a125f-8022-4f5a-9384-e54dd8e13b32"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:48.841557 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.841513 6000 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-workload-certs\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:48.841557 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.841555 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5x69\" (UniqueName: \"kubernetes.io/projected/599a125f-8022-4f5a-9384-e54dd8e13b32-kube-api-access-m5x69\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:48.841821 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.841572 6000 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-podinfo\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:48.841821 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.841585 6000 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-token\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:48.841821 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.841598 6000 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-istio-envoy\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:48.841821 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:48.841611 6000 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/599a125f-8022-4f5a-9384-e54dd8e13b32-credential-socket\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:49.137927 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:49.137889 6000 generic.go:356] "Generic (PLEG): container finished" podID="599a125f-8022-4f5a-9384-e54dd8e13b32" containerID="75507b9736352520d19b589257ee1dd35be205c5986d354d8df2188362afd4b2" exitCode=0 Nov 04 04:01:49.138389 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:49.137947 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" Nov 04 04:01:49.138548 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:49.137936 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" event={"ID":"599a125f-8022-4f5a-9384-e54dd8e13b32","Type":"ContainerDied","Data":"75507b9736352520d19b589257ee1dd35be205c5986d354d8df2188362afd4b2"} Nov 04 04:01:49.138548 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:49.138513 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w" event={"ID":"599a125f-8022-4f5a-9384-e54dd8e13b32","Type":"ContainerDied","Data":"526addb02caab5f265c1ad90713ce1f1dde78d4e1e2930181406c6fe5255f1b8"} Nov 04 04:01:49.138667 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:49.138553 6000 scope.go:122] "RemoveContainer" containerID="75507b9736352520d19b589257ee1dd35be205c5986d354d8df2188362afd4b2" Nov 04 04:01:49.151604 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:49.151587 6000 scope.go:122] "RemoveContainer" containerID="75507b9736352520d19b589257ee1dd35be205c5986d354d8df2188362afd4b2" Nov 04 04:01:49.152112 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:01:49.152092 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75507b9736352520d19b589257ee1dd35be205c5986d354d8df2188362afd4b2\": container with ID starting with 75507b9736352520d19b589257ee1dd35be205c5986d354d8df2188362afd4b2 not found: ID does not exist" containerID="75507b9736352520d19b589257ee1dd35be205c5986d354d8df2188362afd4b2" Nov 04 04:01:49.152199 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:49.152120 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75507b9736352520d19b589257ee1dd35be205c5986d354d8df2188362afd4b2"} err="failed to get container status \"75507b9736352520d19b589257ee1dd35be205c5986d354d8df2188362afd4b2\": rpc error: code = NotFound desc = could not find container \"75507b9736352520d19b589257ee1dd35be205c5986d354d8df2188362afd4b2\": container with ID starting with 75507b9736352520d19b589257ee1dd35be205c5986d354d8df2188362afd4b2 not found: ID does not exist" Nov 04 04:01:49.166852 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:49.166821 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w"] Nov 04 04:01:49.170942 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:49.170916 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-dkf7w"] Nov 04 04:01:50.082607 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.082574 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="599a125f-8022-4f5a-9384-e54dd8e13b32" path="/var/lib/kubelet/pods/599a125f-8022-4f5a-9384-e54dd8e13b32/volumes" Nov 04 04:01:50.526067 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.525980 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-zf6m4"] Nov 04 04:01:50.631311 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.631277 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442"] Nov 04 04:01:50.632140 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632108 6000 cpu_manager.go:414] "containerMap: removing container" podUID="6603cd37-3576-49ff-a87c-e307d4d35706" containerName="main" Nov 04 04:01:50.632318 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632299 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6603cd37-3576-49ff-a87c-e307d4d35706" containerName="main" Nov 04 04:01:50.632318 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632322 6000 cpu_manager.go:414] "containerMap: removing container" podUID="f996ad18-5c32-4945-822e-62ed92214a54" containerName="main" Nov 04 04:01:50.632459 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632331 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f996ad18-5c32-4945-822e-62ed92214a54" containerName="main" Nov 04 04:01:50.632459 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632354 6000 cpu_manager.go:414] "containerMap: removing container" podUID="5858d840-2308-4cf2-b7f3-9f73509bb196" containerName="istio-proxy" Nov 04 04:01:50.632459 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632362 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5858d840-2308-4cf2-b7f3-9f73509bb196" containerName="istio-proxy" Nov 04 04:01:50.632459 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632372 6000 cpu_manager.go:414] "containerMap: removing container" podUID="48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d" containerName="istio-proxy" Nov 04 04:01:50.632459 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632380 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d" containerName="istio-proxy" Nov 04 04:01:50.632459 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632392 6000 cpu_manager.go:414] "containerMap: removing container" podUID="e48d6395-918f-4ebd-91c9-4b93edbb520d" containerName="payload-processing" Nov 04 04:01:50.632459 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632399 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e48d6395-918f-4ebd-91c9-4b93edbb520d" containerName="payload-processing" Nov 04 04:01:50.632459 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632419 6000 cpu_manager.go:414] "containerMap: removing container" podUID="5869a8f0-b341-4616-b258-e5f6c9bab8f5" containerName="istio-proxy" Nov 04 04:01:50.632459 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632427 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5869a8f0-b341-4616-b258-e5f6c9bab8f5" containerName="istio-proxy" Nov 04 04:01:50.632459 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632455 6000 cpu_manager.go:414] "containerMap: removing container" podUID="200578c9-4fac-445d-b697-55d8d79310fb" containerName="payload-pre-processing" Nov 04 04:01:50.632943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632463 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="200578c9-4fac-445d-b697-55d8d79310fb" containerName="payload-pre-processing" Nov 04 04:01:50.632943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632476 6000 cpu_manager.go:414] "containerMap: removing container" podUID="52e771b7-2c73-4b2f-8266-f37cb72490bd" containerName="payload-pre-processing" Nov 04 04:01:50.632943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632484 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="52e771b7-2c73-4b2f-8266-f37cb72490bd" containerName="payload-pre-processing" Nov 04 04:01:50.632943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632495 6000 cpu_manager.go:414] "containerMap: removing container" podUID="599a125f-8022-4f5a-9384-e54dd8e13b32" containerName="istio-proxy" Nov 04 04:01:50.632943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632503 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="599a125f-8022-4f5a-9384-e54dd8e13b32" containerName="istio-proxy" Nov 04 04:01:50.632943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632519 6000 cpu_manager.go:414] "containerMap: removing container" podUID="5858d840-2308-4cf2-b7f3-9f73509bb196" containerName="istio-proxy" Nov 04 04:01:50.632943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632526 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5858d840-2308-4cf2-b7f3-9f73509bb196" containerName="istio-proxy" Nov 04 04:01:50.632943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632639 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="48bb22c3-c4b7-45e3-ace0-b7b33b94fa2d" containerName="istio-proxy" Nov 04 04:01:50.632943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632653 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="e48d6395-918f-4ebd-91c9-4b93edbb520d" containerName="payload-processing" Nov 04 04:01:50.632943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632665 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="5869a8f0-b341-4616-b258-e5f6c9bab8f5" containerName="istio-proxy" Nov 04 04:01:50.632943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632678 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="200578c9-4fac-445d-b697-55d8d79310fb" containerName="payload-pre-processing" Nov 04 04:01:50.632943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632688 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="52e771b7-2c73-4b2f-8266-f37cb72490bd" containerName="payload-pre-processing" Nov 04 04:01:50.632943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632698 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="599a125f-8022-4f5a-9384-e54dd8e13b32" containerName="istio-proxy" Nov 04 04:01:50.632943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632708 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="5858d840-2308-4cf2-b7f3-9f73509bb196" containerName="istio-proxy" Nov 04 04:01:50.632943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632718 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="6603cd37-3576-49ff-a87c-e307d4d35706" containerName="main" Nov 04 04:01:50.632943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632726 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="f996ad18-5c32-4945-822e-62ed92214a54" containerName="main" Nov 04 04:01:50.632943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.632737 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="5858d840-2308-4cf2-b7f3-9f73509bb196" containerName="istio-proxy" Nov 04 04:01:50.638180 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.638149 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.654871 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.654834 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442"] Nov 04 04:01:50.756068 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:01:50.756029 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 04:01:50.767717 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.767682 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl6sx\" (UniqueName: \"kubernetes.io/projected/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-kube-api-access-cl6sx\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.767878 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.767736 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-envoy\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.767878 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.767850 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-credential-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.767944 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.767889 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-workload-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.767944 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.767910 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-token\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.767944 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.767931 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-podinfo\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.768054 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.767990 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istiod-ca-cert\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.768091 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.768053 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-workload-certs\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.768128 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.768088 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-data\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.869780 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.869587 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-data\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.870687 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.870069 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-data\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.870687 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.870314 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl6sx\" (UniqueName: \"kubernetes.io/projected/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-kube-api-access-cl6sx\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.870910 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.870706 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-envoy\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.874815 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.872421 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-credential-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.874815 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.872481 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-workload-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.874815 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.872516 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-token\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.874815 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.872551 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-podinfo\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.874815 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.872595 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istiod-ca-cert\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.874815 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.872652 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-workload-certs\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.874815 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.873057 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-workload-certs\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.874815 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.873299 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-workload-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.875308 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.874865 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istiod-ca-cert\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.875710 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.875680 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-credential-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.880786 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.880713 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-podinfo\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.896158 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.896120 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-envoy\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.943217 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.943184 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl6sx\" (UniqueName: \"kubernetes.io/projected/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-kube-api-access-cl6sx\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.946303 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.946270 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-token\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:50.959335 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:50.959302 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:51.255526 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:51.255496 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442"] Nov 04 04:01:51.257238 ip-10-0-129-134 kubenswrapper[6000]: W1104 04:01:51.257208 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1aa05bb_e6a2_49bd_96b3_ec4122ba1391.slice/crio-5315afd663bce289671d41a0cf61c198906c535a543bb442991e8f7a45527661 WatchSource:0}: Error finding container 5315afd663bce289671d41a0cf61c198906c535a543bb442991e8f7a45527661: Status 404 returned error can't find the container with id 5315afd663bce289671d41a0cf61c198906c535a543bb442991e8f7a45527661 Nov 04 04:01:51.260279 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:51.260240 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:01:51.260369 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:51.260328 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:01:51.260411 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:51.260377 6000 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:01:51.515839 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:51.514956 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" podUID="eef5b758-9117-4d9a-901c-29f03be11804" containerName="main" containerID="cri-o://f3d40f3ccaff13b2fe489adbd24cde10adcba58f226771edacaae160b6802440" gracePeriod=45 Nov 04 04:01:51.527399 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:51.527358 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" podUID="785d143f-077d-4ebd-8b1f-1cd3c67aac47" containerName="main" containerID="cri-o://f65be9cef0aa4ca34fce40cbda1fc4810c525f886995a7e75c9e3182a145e3bb" gracePeriod=45 Nov 04 04:01:51.541512 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:51.541469 6000 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" podUID="eef5b758-9117-4d9a-901c-29f03be11804" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": dial tcp 10.133.0.53:8000: connect: connection refused" Nov 04 04:01:51.989653 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:51.988717 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 04:01:51.994304 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:51.993634 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 04:01:52.198930 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.153401 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eef5b758-9117-4d9a-901c-29f03be11804-tls-certs\" (UniqueName: \"kubernetes.io/secret/eef5b758-9117-4d9a-901c-29f03be11804-tls-certs\") pod \"eef5b758-9117-4d9a-901c-29f03be11804\" (UID: \"eef5b758-9117-4d9a-901c-29f03be11804\") " Nov 04 04:01:52.198930 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.153474 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-tmp-dir\") pod \"eef5b758-9117-4d9a-901c-29f03be11804\" (UID: \"eef5b758-9117-4d9a-901c-29f03be11804\") " Nov 04 04:01:52.198930 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.153566 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eef5b758-9117-4d9a-901c-29f03be11804-kube-api-access-vg4n2\" (UniqueName: \"kubernetes.io/projected/eef5b758-9117-4d9a-901c-29f03be11804-kube-api-access-vg4n2\") pod \"eef5b758-9117-4d9a-901c-29f03be11804\" (UID: \"eef5b758-9117-4d9a-901c-29f03be11804\") " Nov 04 04:01:52.198930 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.153630 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/785d143f-077d-4ebd-8b1f-1cd3c67aac47-kube-api-access-9w54g\" (UniqueName: \"kubernetes.io/projected/785d143f-077d-4ebd-8b1f-1cd3c67aac47-kube-api-access-9w54g\") pod \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\" (UID: \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\") " Nov 04 04:01:52.198930 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.153672 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-model-cache\") pod \"eef5b758-9117-4d9a-901c-29f03be11804\" (UID: \"eef5b758-9117-4d9a-901c-29f03be11804\") " Nov 04 04:01:52.198930 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.153715 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-dshm\" (UniqueName: \"kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-dshm\") pod \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\" (UID: \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\") " Nov 04 04:01:52.198930 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.153803 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-tmp-dir\") pod \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\" (UID: \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\") " Nov 04 04:01:52.198930 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.153850 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-model-cache\") pod \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\" (UID: \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\") " Nov 04 04:01:52.198930 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.153889 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-home\" (UniqueName: \"kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-home\") pod \"eef5b758-9117-4d9a-901c-29f03be11804\" (UID: \"eef5b758-9117-4d9a-901c-29f03be11804\") " Nov 04 04:01:52.198930 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.153951 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-dshm\" (UniqueName: \"kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-dshm\") pod \"eef5b758-9117-4d9a-901c-29f03be11804\" (UID: \"eef5b758-9117-4d9a-901c-29f03be11804\") " Nov 04 04:01:52.198930 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.154030 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/785d143f-077d-4ebd-8b1f-1cd3c67aac47-tls-certs\" (UniqueName: \"kubernetes.io/secret/785d143f-077d-4ebd-8b1f-1cd3c67aac47-tls-certs\") pod \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\" (UID: \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\") " Nov 04 04:01:52.198930 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.154065 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-home\" (UniqueName: \"kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-home\") pod \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\" (UID: \"785d143f-077d-4ebd-8b1f-1cd3c67aac47\") " Nov 04 04:01:52.198930 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.171904 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-home" pod "eef5b758-9117-4d9a-901c-29f03be11804" (UID: "eef5b758-9117-4d9a-901c-29f03be11804"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:52.198930 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.172120 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-tmp-dir" pod "785d143f-077d-4ebd-8b1f-1cd3c67aac47" (UID: "785d143f-077d-4ebd-8b1f-1cd3c67aac47"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:52.198930 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.172286 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-model-cache" pod "785d143f-077d-4ebd-8b1f-1cd3c67aac47" (UID: "785d143f-077d-4ebd-8b1f-1cd3c67aac47"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:52.198930 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.186786 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-home" pod "785d143f-077d-4ebd-8b1f-1cd3c67aac47" (UID: "785d143f-077d-4ebd-8b1f-1cd3c67aac47"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:52.201899 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.201857 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-dshm" pod "eef5b758-9117-4d9a-901c-29f03be11804" (UID: "eef5b758-9117-4d9a-901c-29f03be11804"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:52.202176 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.202148 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-model-cache" pod "eef5b758-9117-4d9a-901c-29f03be11804" (UID: "eef5b758-9117-4d9a-901c-29f03be11804"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:52.204323 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.202929 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-tmp-dir" pod "eef5b758-9117-4d9a-901c-29f03be11804" (UID: "eef5b758-9117-4d9a-901c-29f03be11804"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:52.289821 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.288579 6000 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-model-cache\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:52.289821 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.288621 6000 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-tmp-dir\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:52.289821 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.288636 6000 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-model-cache\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:52.289821 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.288650 6000 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-home\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:52.289821 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.288664 6000 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-dshm\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:52.289821 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.288682 6000 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-home\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:52.289821 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.288695 6000 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eef5b758-9117-4d9a-901c-29f03be11804-tmp-dir\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:52.301820 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.294959 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eef5b758-9117-4d9a-901c-29f03be11804-tls-certs" pod "eef5b758-9117-4d9a-901c-29f03be11804" (UID: "eef5b758-9117-4d9a-901c-29f03be11804"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 04:01:52.306889 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.306832 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-dshm" pod "785d143f-077d-4ebd-8b1f-1cd3c67aac47" (UID: "785d143f-077d-4ebd-8b1f-1cd3c67aac47"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:01:52.307062 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.306903 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/785d143f-077d-4ebd-8b1f-1cd3c67aac47-tls-certs" pod "785d143f-077d-4ebd-8b1f-1cd3c67aac47" (UID: "785d143f-077d-4ebd-8b1f-1cd3c67aac47"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 04:01:52.337734 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.337689 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/785d143f-077d-4ebd-8b1f-1cd3c67aac47-kube-api-access-9w54g" pod "785d143f-077d-4ebd-8b1f-1cd3c67aac47" (UID: "785d143f-077d-4ebd-8b1f-1cd3c67aac47"). InnerVolumeSpecName "kube-api-access-9w54g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:01:52.371891 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.371845 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eef5b758-9117-4d9a-901c-29f03be11804-kube-api-access-vg4n2" pod "eef5b758-9117-4d9a-901c-29f03be11804" (UID: "eef5b758-9117-4d9a-901c-29f03be11804"). InnerVolumeSpecName "kube-api-access-vg4n2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:01:52.384039 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.375467 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" event={"ID":"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391","Type":"ContainerStarted","Data":"c63cebc4a4651159dc6c37458c190548483d0cb937e7b75ecf12a273cbf361fe"} Nov 04 04:01:52.384039 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.375530 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" event={"ID":"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391","Type":"ContainerStarted","Data":"5315afd663bce289671d41a0cf61c198906c535a543bb442991e8f7a45527661"} Nov 04 04:01:52.384039 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.375556 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s"] Nov 04 04:01:52.384039 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.375846 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s" podUID="97aca100-541e-48ff-9b0e-5d4853b700ff" containerName="maas-api" containerID="cri-o://efa88804b1b545a3dcdb5e1918bd7bf7383e5b2a5538fadd6cfcd9b0feafbec0" gracePeriod=30 Nov 04 04:01:52.390942 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.390822 6000 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/785d143f-077d-4ebd-8b1f-1cd3c67aac47-dshm\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:52.391190 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.391173 6000 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/785d143f-077d-4ebd-8b1f-1cd3c67aac47-tls-certs\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:52.391313 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.391296 6000 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eef5b758-9117-4d9a-901c-29f03be11804-tls-certs\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:52.391501 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.391485 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vg4n2\" (UniqueName: \"kubernetes.io/projected/eef5b758-9117-4d9a-901c-29f03be11804-kube-api-access-vg4n2\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:52.391617 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.391604 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9w54g\" (UniqueName: \"kubernetes.io/projected/785d143f-077d-4ebd-8b1f-1cd3c67aac47-kube-api-access-9w54g\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:52.393565 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.393533 6000 generic.go:356] "Generic (PLEG): container finished" podID="785d143f-077d-4ebd-8b1f-1cd3c67aac47" containerID="f65be9cef0aa4ca34fce40cbda1fc4810c525f886995a7e75c9e3182a145e3bb" exitCode=0 Nov 04 04:01:52.393825 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.393803 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" event={"ID":"785d143f-077d-4ebd-8b1f-1cd3c67aac47","Type":"ContainerDied","Data":"f65be9cef0aa4ca34fce40cbda1fc4810c525f886995a7e75c9e3182a145e3bb"} Nov 04 04:01:52.394855 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.394823 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" event={"ID":"785d143f-077d-4ebd-8b1f-1cd3c67aac47","Type":"ContainerDied","Data":"e6818145ded7218693a752031fc0eeadc04eabbbd995a51d575b24d0f0c11621"} Nov 04 04:01:52.394992 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.394978 6000 scope.go:122] "RemoveContainer" containerID="f65be9cef0aa4ca34fce40cbda1fc4810c525f886995a7e75c9e3182a145e3bb" Nov 04 04:01:52.395260 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.395244 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh" Nov 04 04:01:52.423383 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.420851 6000 generic.go:356] "Generic (PLEG): container finished" podID="eef5b758-9117-4d9a-901c-29f03be11804" containerID="f3d40f3ccaff13b2fe489adbd24cde10adcba58f226771edacaae160b6802440" exitCode=0 Nov 04 04:01:52.423383 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.420956 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" event={"ID":"eef5b758-9117-4d9a-901c-29f03be11804","Type":"ContainerDied","Data":"f3d40f3ccaff13b2fe489adbd24cde10adcba58f226771edacaae160b6802440"} Nov 04 04:01:52.423383 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.420993 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" event={"ID":"eef5b758-9117-4d9a-901c-29f03be11804","Type":"ContainerDied","Data":"295b5ab5a53b2aea3223670ea16eb053ce737dd9c2ba4ce5c9db53acda274276"} Nov 04 04:01:52.423383 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.421082 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b" Nov 04 04:01:52.530788 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.524830 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" podStartSLOduration=2.524806773 podStartE2EDuration="2.524806773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:01:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:01:52.406356128 +0000 UTC m=+710.444796105" watchObservedRunningTime="2026-11-04 04:01:52.524806773 +0000 UTC m=+710.563246738" Nov 04 04:01:52.626250 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.621598 6000 scope.go:122] "RemoveContainer" containerID="f65be9cef0aa4ca34fce40cbda1fc4810c525f886995a7e75c9e3182a145e3bb" Nov 04 04:01:52.640192 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:01:52.639841 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f65be9cef0aa4ca34fce40cbda1fc4810c525f886995a7e75c9e3182a145e3bb\": container with ID starting with f65be9cef0aa4ca34fce40cbda1fc4810c525f886995a7e75c9e3182a145e3bb not found: ID does not exist" containerID="f65be9cef0aa4ca34fce40cbda1fc4810c525f886995a7e75c9e3182a145e3bb" Nov 04 04:01:52.640192 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.639893 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f65be9cef0aa4ca34fce40cbda1fc4810c525f886995a7e75c9e3182a145e3bb"} err="failed to get container status \"f65be9cef0aa4ca34fce40cbda1fc4810c525f886995a7e75c9e3182a145e3bb\": rpc error: code = NotFound desc = could not find container \"f65be9cef0aa4ca34fce40cbda1fc4810c525f886995a7e75c9e3182a145e3bb\": container with ID starting with f65be9cef0aa4ca34fce40cbda1fc4810c525f886995a7e75c9e3182a145e3bb not found: ID does not exist" Nov 04 04:01:52.640192 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.639921 6000 scope.go:122] "RemoveContainer" containerID="f3d40f3ccaff13b2fe489adbd24cde10adcba58f226771edacaae160b6802440" Nov 04 04:01:52.646771 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.646700 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b"] Nov 04 04:01:52.729062 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.728966 6000 scope.go:122] "RemoveContainer" containerID="f3d40f3ccaff13b2fe489adbd24cde10adcba58f226771edacaae160b6802440" Nov 04 04:01:52.730270 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:01:52.730240 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3d40f3ccaff13b2fe489adbd24cde10adcba58f226771edacaae160b6802440\": container with ID starting with f3d40f3ccaff13b2fe489adbd24cde10adcba58f226771edacaae160b6802440 not found: ID does not exist" containerID="f3d40f3ccaff13b2fe489adbd24cde10adcba58f226771edacaae160b6802440" Nov 04 04:01:52.730448 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.730421 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3d40f3ccaff13b2fe489adbd24cde10adcba58f226771edacaae160b6802440"} err="failed to get container status \"f3d40f3ccaff13b2fe489adbd24cde10adcba58f226771edacaae160b6802440\": rpc error: code = NotFound desc = could not find container \"f3d40f3ccaff13b2fe489adbd24cde10adcba58f226771edacaae160b6802440\": container with ID starting with f3d40f3ccaff13b2fe489adbd24cde10adcba58f226771edacaae160b6802440 not found: ID does not exist" Nov 04 04:01:52.810933 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.810810 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-4afe0e/premium-w0-4afe0e-kserve-6c5c95b95c-njj8b"] Nov 04 04:01:52.898290 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.891802 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh"] Nov 04 04:01:52.898290 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.898198 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-4afe0e/simulator-w0-4afe0e-kserve-7d6784c4fb-8kkfh"] Nov 04 04:01:52.961860 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.961161 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:01:52.983990 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.981257 6000 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Nov 04 04:01:52.983990 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:52.981332 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" podUID="c1aa05bb-e6a2-49bd-96b3-ec4122ba1391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Nov 04 04:01:53.576827 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:53.572079 6000 generic.go:356] "Generic (PLEG): container finished" podID="97aca100-541e-48ff-9b0e-5d4853b700ff" containerID="efa88804b1b545a3dcdb5e1918bd7bf7383e5b2a5538fadd6cfcd9b0feafbec0" exitCode=0 Nov 04 04:01:53.581724 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:53.581683 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s" event={"ID":"97aca100-541e-48ff-9b0e-5d4853b700ff","Type":"ContainerDied","Data":"efa88804b1b545a3dcdb5e1918bd7bf7383e5b2a5538fadd6cfcd9b0feafbec0"} Nov 04 04:01:53.671122 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:53.671087 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s" Nov 04 04:01:53.836009 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:53.834733 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/97aca100-541e-48ff-9b0e-5d4853b700ff-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97aca100-541e-48ff-9b0e-5d4853b700ff-maas-api-tls\") pod \"97aca100-541e-48ff-9b0e-5d4853b700ff\" (UID: \"97aca100-541e-48ff-9b0e-5d4853b700ff\") " Nov 04 04:01:53.836009 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:53.834883 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97aca100-541e-48ff-9b0e-5d4853b700ff-kube-api-access-d2gv9\" (UniqueName: \"kubernetes.io/projected/97aca100-541e-48ff-9b0e-5d4853b700ff-kube-api-access-d2gv9\") pod \"97aca100-541e-48ff-9b0e-5d4853b700ff\" (UID: \"97aca100-541e-48ff-9b0e-5d4853b700ff\") " Nov 04 04:01:53.836009 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:53.834980 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/97aca100-541e-48ff-9b0e-5d4853b700ff-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97aca100-541e-48ff-9b0e-5d4853b700ff-openshift-service-ca\") pod \"97aca100-541e-48ff-9b0e-5d4853b700ff\" (UID: \"97aca100-541e-48ff-9b0e-5d4853b700ff\") " Nov 04 04:01:53.836009 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:53.835032 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/97aca100-541e-48ff-9b0e-5d4853b700ff-metrics-certs\" (UniqueName: \"kubernetes.io/secret/97aca100-541e-48ff-9b0e-5d4853b700ff-metrics-certs\") pod \"97aca100-541e-48ff-9b0e-5d4853b700ff\" (UID: \"97aca100-541e-48ff-9b0e-5d4853b700ff\") " Nov 04 04:01:53.837074 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:53.837023 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97aca100-541e-48ff-9b0e-5d4853b700ff-openshift-service-ca" pod "97aca100-541e-48ff-9b0e-5d4853b700ff" (UID: "97aca100-541e-48ff-9b0e-5d4853b700ff"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:01:53.885169 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:53.882999 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97aca100-541e-48ff-9b0e-5d4853b700ff-kube-api-access-d2gv9" pod "97aca100-541e-48ff-9b0e-5d4853b700ff" (UID: "97aca100-541e-48ff-9b0e-5d4853b700ff"). InnerVolumeSpecName "kube-api-access-d2gv9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:01:53.894489 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:53.894348 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97aca100-541e-48ff-9b0e-5d4853b700ff-metrics-certs" pod "97aca100-541e-48ff-9b0e-5d4853b700ff" (UID: "97aca100-541e-48ff-9b0e-5d4853b700ff"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 04:01:53.902880 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:53.902780 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97aca100-541e-48ff-9b0e-5d4853b700ff-maas-api-tls" pod "97aca100-541e-48ff-9b0e-5d4853b700ff" (UID: "97aca100-541e-48ff-9b0e-5d4853b700ff"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 04:01:53.938302 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:53.938180 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2gv9\" (UniqueName: \"kubernetes.io/projected/97aca100-541e-48ff-9b0e-5d4853b700ff-kube-api-access-d2gv9\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:53.938302 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:53.938224 6000 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97aca100-541e-48ff-9b0e-5d4853b700ff-openshift-service-ca\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:53.938302 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:53.938243 6000 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97aca100-541e-48ff-9b0e-5d4853b700ff-metrics-certs\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:53.938302 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:53.938257 6000 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97aca100-541e-48ff-9b0e-5d4853b700ff-maas-api-tls\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:53.962658 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:53.962170 6000 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Nov 04 04:01:53.962658 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:53.962262 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" podUID="c1aa05bb-e6a2-49bd-96b3-ec4122ba1391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Nov 04 04:01:54.082303 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:54.082268 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="785d143f-077d-4ebd-8b1f-1cd3c67aac47" path="/var/lib/kubelet/pods/785d143f-077d-4ebd-8b1f-1cd3c67aac47/volumes" Nov 04 04:01:54.082698 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:54.082684 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eef5b758-9117-4d9a-901c-29f03be11804" path="/var/lib/kubelet/pods/eef5b758-9117-4d9a-901c-29f03be11804/volumes" Nov 04 04:01:54.639043 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:54.638332 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s" Nov 04 04:01:54.639043 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:54.638359 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s" event={"ID":"97aca100-541e-48ff-9b0e-5d4853b700ff","Type":"ContainerDied","Data":"e881be6f353687fbe6c2c9e4e99c044eba88770acc35ba41f2db1d456b8610c2"} Nov 04 04:01:54.639043 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:54.638454 6000 scope.go:122] "RemoveContainer" containerID="efa88804b1b545a3dcdb5e1918bd7bf7383e5b2a5538fadd6cfcd9b0feafbec0" Nov 04 04:01:54.660110 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:54.660075 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s"] Nov 04 04:01:54.664003 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:54.663974 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-64c20d-786c85dfb4-7756s"] Nov 04 04:01:54.959863 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:54.959748 6000 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Nov 04 04:01:54.959863 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:54.959832 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" podUID="c1aa05bb-e6a2-49bd-96b3-ec4122ba1391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Nov 04 04:01:55.961200 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:55.961158 6000 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Nov 04 04:01:55.962370 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:55.961884 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" podUID="c1aa05bb-e6a2-49bd-96b3-ec4122ba1391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Nov 04 04:01:56.081315 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:56.081252 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="97aca100-541e-48ff-9b0e-5d4853b700ff" path="/var/lib/kubelet/pods/97aca100-541e-48ff-9b0e-5d4853b700ff/volumes" Nov 04 04:01:56.960376 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:56.960337 6000 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Nov 04 04:01:56.960586 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:56.960425 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" podUID="c1aa05bb-e6a2-49bd-96b3-ec4122ba1391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Nov 04 04:01:57.272273 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:57.272244 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l" Nov 04 04:01:57.274171 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:57.274148 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/31648502-f58d-43a9-a746-bcfb74eb70f7-kube-api-access-xcp78\" (UniqueName: \"kubernetes.io/projected/31648502-f58d-43a9-a746-bcfb74eb70f7-kube-api-access-xcp78\") pod \"31648502-f58d-43a9-a746-bcfb74eb70f7\" (UID: \"31648502-f58d-43a9-a746-bcfb74eb70f7\") " Nov 04 04:01:57.274276 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:57.274263 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/31648502-f58d-43a9-a746-bcfb74eb70f7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31648502-f58d-43a9-a746-bcfb74eb70f7-plugins-config-volume\") pod \"31648502-f58d-43a9-a746-bcfb74eb70f7\" (UID: \"31648502-f58d-43a9-a746-bcfb74eb70f7\") " Nov 04 04:01:57.274648 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:57.274612 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31648502-f58d-43a9-a746-bcfb74eb70f7-plugins-config-volume" pod "31648502-f58d-43a9-a746-bcfb74eb70f7" (UID: "31648502-f58d-43a9-a746-bcfb74eb70f7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:01:57.276249 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:57.276219 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31648502-f58d-43a9-a746-bcfb74eb70f7-kube-api-access-xcp78" pod "31648502-f58d-43a9-a746-bcfb74eb70f7" (UID: "31648502-f58d-43a9-a746-bcfb74eb70f7"). InnerVolumeSpecName "kube-api-access-xcp78". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:01:57.374968 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:57.374930 6000 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31648502-f58d-43a9-a746-bcfb74eb70f7-plugins-config-volume\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:57.374968 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:57.374960 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcp78\" (UniqueName: \"kubernetes.io/projected/31648502-f58d-43a9-a746-bcfb74eb70f7-kube-api-access-xcp78\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:01:57.669765 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:57.669706 6000 generic.go:356] "Generic (PLEG): container finished" podID="31648502-f58d-43a9-a746-bcfb74eb70f7" containerID="225646120e3d005645cd4800c05f154e4909d80f60de719df674448ad3021f91" exitCode=0 Nov 04 04:01:57.669765 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:57.669749 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l" Nov 04 04:01:57.669991 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:57.669791 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l" event={"ID":"31648502-f58d-43a9-a746-bcfb74eb70f7","Type":"ContainerDied","Data":"225646120e3d005645cd4800c05f154e4909d80f60de719df674448ad3021f91"} Nov 04 04:01:57.669991 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:57.669864 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l" event={"ID":"31648502-f58d-43a9-a746-bcfb74eb70f7","Type":"ContainerDied","Data":"7761f255fa61a255e285d6f5d35bbced486d8b1a2da5c96b4db19729d09bde33"} Nov 04 04:01:57.669991 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:57.669890 6000 scope.go:122] "RemoveContainer" containerID="225646120e3d005645cd4800c05f154e4909d80f60de719df674448ad3021f91" Nov 04 04:01:57.679945 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:57.679921 6000 scope.go:122] "RemoveContainer" containerID="225646120e3d005645cd4800c05f154e4909d80f60de719df674448ad3021f91" Nov 04 04:01:57.680263 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:01:57.680244 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"225646120e3d005645cd4800c05f154e4909d80f60de719df674448ad3021f91\": container with ID starting with 225646120e3d005645cd4800c05f154e4909d80f60de719df674448ad3021f91 not found: ID does not exist" containerID="225646120e3d005645cd4800c05f154e4909d80f60de719df674448ad3021f91" Nov 04 04:01:57.680348 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:57.680271 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"225646120e3d005645cd4800c05f154e4909d80f60de719df674448ad3021f91"} err="failed to get container status \"225646120e3d005645cd4800c05f154e4909d80f60de719df674448ad3021f91\": rpc error: code = NotFound desc = could not find container \"225646120e3d005645cd4800c05f154e4909d80f60de719df674448ad3021f91\": container with ID starting with 225646120e3d005645cd4800c05f154e4909d80f60de719df674448ad3021f91 not found: ID does not exist" Nov 04 04:01:57.693154 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:57.693117 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l"] Nov 04 04:01:57.697412 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:57.697381 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-56a44f-576768df8d-ddt9l"] Nov 04 04:01:57.960180 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:57.960076 6000 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Nov 04 04:01:57.960180 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:57.960140 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" podUID="c1aa05bb-e6a2-49bd-96b3-ec4122ba1391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Nov 04 04:01:58.080628 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:01:58.080596 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="31648502-f58d-43a9-a746-bcfb74eb70f7" path="/var/lib/kubelet/pods/31648502-f58d-43a9-a746-bcfb74eb70f7/volumes" Nov 04 04:02:00.002937 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:00.002898 6000 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 04:02:00.003440 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:00.002974 6000 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" podUID="c1aa05bb-e6a2-49bd-96b3-ec4122ba1391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 04:02:00.480100 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:00.480065 6000 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:02:00.480374 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:00.480354 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:02:00.481082 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:00.481061 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:02:05.205017 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:05.201244 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c"] Nov 04 04:02:05.205017 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:05.201512 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" podUID="78a5d7c4-f484-4788-a260-ead9c2ba30da" containerName="istio-proxy" containerID="cri-o://2e521d439765f43cc64e1d1576e3eeb17278594e5829e50a9f4865b57eff4de9" gracePeriod=30 Nov 04 04:02:05.213932 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:05.213893 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m"] Nov 04 04:02:05.218567 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:05.218504 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" podUID="f15a8b2c-4f91-4792-869d-26e339189978" containerName="istio-proxy" containerID="cri-o://22407c01e6b73c77b5e3523d310725fb5f996d5ddb360a572cda1a492f94f20d" gracePeriod=30 Nov 04 04:02:07.152814 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:07.152772 6000 patch_prober.go:29] interesting pod/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 04:02:07.153295 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:07.152855 6000 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" podUID="f15a8b2c-4f91-4792-869d-26e339189978" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 04:02:07.581742 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:07.581703 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s"] Nov 04 04:02:07.582261 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:07.582225 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" podUID="4afd26c6-87c7-4e7e-a315-5e4106ce8a21" containerName="istio-proxy" containerID="cri-o://a7bc59fec6b43e18b21b77cb26a6de51a2c010128600ece5a62aa2373eb0db96" gracePeriod=30 Nov 04 04:02:07.587183 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:07.587139 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442"] Nov 04 04:02:07.587407 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:07.587378 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" podUID="c1aa05bb-e6a2-49bd-96b3-ec4122ba1391" containerName="istio-proxy" containerID="cri-o://c63cebc4a4651159dc6c37458c190548483d0cb937e7b75ecf12a273cbf361fe" gracePeriod=30 Nov 04 04:02:07.589114 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:07.589090 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv"] Nov 04 04:02:07.589326 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:07.589289 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" podUID="a8354d1d-fe79-42ca-b453-e8b315d4b9a8" containerName="istio-proxy" containerID="cri-o://7400917a30dea2e8b404edff1f038799c72a46b0174da8545b6db2750a64f8a5" gracePeriod=30 Nov 04 04:02:07.906307 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:07.906210 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-zf6m4"] Nov 04 04:02:10.490351 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.490275 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:02:10.533748 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.533717 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 04:02:10.534066 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.534044 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-podinfo\") pod \"78a5d7c4-f484-4788-a260-ead9c2ba30da\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " Nov 04 04:02:10.534172 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.534091 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-envoy\") pod \"78a5d7c4-f484-4788-a260-ead9c2ba30da\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " Nov 04 04:02:10.534172 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.534132 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-token\" (UniqueName: \"kubernetes.io/projected/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-token\") pod \"78a5d7c4-f484-4788-a260-ead9c2ba30da\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " Nov 04 04:02:10.534272 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.534186 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-data\") pod \"78a5d7c4-f484-4788-a260-ead9c2ba30da\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " Nov 04 04:02:10.534272 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.534237 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78a5d7c4-f484-4788-a260-ead9c2ba30da-kube-api-access-9bxmx\" (UniqueName: \"kubernetes.io/projected/78a5d7c4-f484-4788-a260-ead9c2ba30da-kube-api-access-9bxmx\") pod \"78a5d7c4-f484-4788-a260-ead9c2ba30da\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " Nov 04 04:02:10.534428 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.534274 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-workload-socket\") pod \"78a5d7c4-f484-4788-a260-ead9c2ba30da\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " Nov 04 04:02:10.534428 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.534305 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-workload-certs\") pod \"78a5d7c4-f484-4788-a260-ead9c2ba30da\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " Nov 04 04:02:10.534428 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.534357 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/78a5d7c4-f484-4788-a260-ead9c2ba30da-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78a5d7c4-f484-4788-a260-ead9c2ba30da-istiod-ca-cert\") pod \"78a5d7c4-f484-4788-a260-ead9c2ba30da\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " Nov 04 04:02:10.534428 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.534412 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-credential-socket\") pod \"78a5d7c4-f484-4788-a260-ead9c2ba30da\" (UID: \"78a5d7c4-f484-4788-a260-ead9c2ba30da\") " Nov 04 04:02:10.534724 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.534679 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-workload-socket" pod "78a5d7c4-f484-4788-a260-ead9c2ba30da" (UID: "78a5d7c4-f484-4788-a260-ead9c2ba30da"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:10.534809 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.534780 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-data" pod "78a5d7c4-f484-4788-a260-ead9c2ba30da" (UID: "78a5d7c4-f484-4788-a260-ead9c2ba30da"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:10.534885 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.534866 6000 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-workload-socket\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:10.534973 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.534939 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-credential-socket" pod "78a5d7c4-f484-4788-a260-ead9c2ba30da" (UID: "78a5d7c4-f484-4788-a260-ead9c2ba30da"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:10.535031 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.534998 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-workload-certs" pod "78a5d7c4-f484-4788-a260-ead9c2ba30da" (UID: "78a5d7c4-f484-4788-a260-ead9c2ba30da"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:10.535251 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.535230 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78a5d7c4-f484-4788-a260-ead9c2ba30da-istiod-ca-cert" pod "78a5d7c4-f484-4788-a260-ead9c2ba30da" (UID: "78a5d7c4-f484-4788-a260-ead9c2ba30da"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:02:10.536982 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.536935 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-envoy" pod "78a5d7c4-f484-4788-a260-ead9c2ba30da" (UID: "78a5d7c4-f484-4788-a260-ead9c2ba30da"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:10.537082 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.537017 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-token" pod "78a5d7c4-f484-4788-a260-ead9c2ba30da" (UID: "78a5d7c4-f484-4788-a260-ead9c2ba30da"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:02:10.537531 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.537500 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78a5d7c4-f484-4788-a260-ead9c2ba30da-kube-api-access-9bxmx" pod "78a5d7c4-f484-4788-a260-ead9c2ba30da" (UID: "78a5d7c4-f484-4788-a260-ead9c2ba30da"). InnerVolumeSpecName "kube-api-access-9bxmx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:02:10.537688 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.537669 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-podinfo" pod "78a5d7c4-f484-4788-a260-ead9c2ba30da" (UID: "78a5d7c4-f484-4788-a260-ead9c2ba30da"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 04:02:10.635222 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.635180 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-workload-certs\") pod \"f15a8b2c-4f91-4792-869d-26e339189978\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " Nov 04 04:02:10.635423 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.635243 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f15a8b2c-4f91-4792-869d-26e339189978-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f15a8b2c-4f91-4792-869d-26e339189978-istio-podinfo\") pod \"f15a8b2c-4f91-4792-869d-26e339189978\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " Nov 04 04:02:10.635423 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.635275 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-credential-socket\") pod \"f15a8b2c-4f91-4792-869d-26e339189978\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " Nov 04 04:02:10.635423 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.635344 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f15a8b2c-4f91-4792-869d-26e339189978-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f15a8b2c-4f91-4792-869d-26e339189978-istiod-ca-cert\") pod \"f15a8b2c-4f91-4792-869d-26e339189978\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " Nov 04 04:02:10.635423 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.635383 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-workload-socket\") pod \"f15a8b2c-4f91-4792-869d-26e339189978\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " Nov 04 04:02:10.635647 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.635426 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f15a8b2c-4f91-4792-869d-26e339189978-istio-token\" (UniqueName: \"kubernetes.io/projected/f15a8b2c-4f91-4792-869d-26e339189978-istio-token\") pod \"f15a8b2c-4f91-4792-869d-26e339189978\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " Nov 04 04:02:10.635647 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.635471 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-istio-data\") pod \"f15a8b2c-4f91-4792-869d-26e339189978\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " Nov 04 04:02:10.635647 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.635530 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f15a8b2c-4f91-4792-869d-26e339189978-kube-api-access-wrl9t\" (UniqueName: \"kubernetes.io/projected/f15a8b2c-4f91-4792-869d-26e339189978-kube-api-access-wrl9t\") pod \"f15a8b2c-4f91-4792-869d-26e339189978\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " Nov 04 04:02:10.635647 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.635530 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-workload-certs" pod "f15a8b2c-4f91-4792-869d-26e339189978" (UID: "f15a8b2c-4f91-4792-869d-26e339189978"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:10.635647 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.635564 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-istio-envoy\") pod \"f15a8b2c-4f91-4792-869d-26e339189978\" (UID: \"f15a8b2c-4f91-4792-869d-26e339189978\") " Nov 04 04:02:10.635913 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.635655 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-credential-socket" pod "f15a8b2c-4f91-4792-869d-26e339189978" (UID: "f15a8b2c-4f91-4792-869d-26e339189978"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:10.635913 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.635877 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-workload-socket" pod "f15a8b2c-4f91-4792-869d-26e339189978" (UID: "f15a8b2c-4f91-4792-869d-26e339189978"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:10.635913 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.635878 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f15a8b2c-4f91-4792-869d-26e339189978-istiod-ca-cert" pod "f15a8b2c-4f91-4792-869d-26e339189978" (UID: "f15a8b2c-4f91-4792-869d-26e339189978"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:02:10.636073 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.635996 6000 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f15a8b2c-4f91-4792-869d-26e339189978-istiod-ca-cert\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:10.636073 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.636015 6000 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-workload-socket\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:10.636073 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.636030 6000 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-podinfo\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:10.636073 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.636044 6000 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-envoy\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:10.636073 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.636057 6000 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-token\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:10.636073 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.636070 6000 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-istio-data\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:10.636374 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.636084 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bxmx\" (UniqueName: \"kubernetes.io/projected/78a5d7c4-f484-4788-a260-ead9c2ba30da-kube-api-access-9bxmx\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:10.636374 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.636099 6000 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-workload-certs\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:10.636374 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.636107 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-istio-data" pod "f15a8b2c-4f91-4792-869d-26e339189978" (UID: "f15a8b2c-4f91-4792-869d-26e339189978"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:10.636374 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.636113 6000 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-workload-certs\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:10.636374 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.636157 6000 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78a5d7c4-f484-4788-a260-ead9c2ba30da-istiod-ca-cert\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:10.636374 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.636177 6000 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78a5d7c4-f484-4788-a260-ead9c2ba30da-credential-socket\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:10.636374 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.636191 6000 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-credential-socket\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:10.637721 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.637693 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f15a8b2c-4f91-4792-869d-26e339189978-istio-podinfo" pod "f15a8b2c-4f91-4792-869d-26e339189978" (UID: "f15a8b2c-4f91-4792-869d-26e339189978"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 04:02:10.638112 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.638095 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f15a8b2c-4f91-4792-869d-26e339189978-istio-token" pod "f15a8b2c-4f91-4792-869d-26e339189978" (UID: "f15a8b2c-4f91-4792-869d-26e339189978"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:02:10.638297 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.638275 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-istio-envoy" pod "f15a8b2c-4f91-4792-869d-26e339189978" (UID: "f15a8b2c-4f91-4792-869d-26e339189978"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:10.638397 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.638353 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f15a8b2c-4f91-4792-869d-26e339189978-kube-api-access-wrl9t" pod "f15a8b2c-4f91-4792-869d-26e339189978" (UID: "f15a8b2c-4f91-4792-869d-26e339189978"). InnerVolumeSpecName "kube-api-access-wrl9t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:02:10.737373 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.737334 6000 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f15a8b2c-4f91-4792-869d-26e339189978-istio-podinfo\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:10.737373 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.737372 6000 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f15a8b2c-4f91-4792-869d-26e339189978-istio-token\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:10.737566 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.737387 6000 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-istio-data\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:10.737566 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.737403 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrl9t\" (UniqueName: \"kubernetes.io/projected/f15a8b2c-4f91-4792-869d-26e339189978-kube-api-access-wrl9t\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:10.737566 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.737419 6000 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f15a8b2c-4f91-4792-869d-26e339189978-istio-envoy\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:10.794683 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.794647 6000 generic.go:356] "Generic (PLEG): container finished" podID="f15a8b2c-4f91-4792-869d-26e339189978" containerID="22407c01e6b73c77b5e3523d310725fb5f996d5ddb360a572cda1a492f94f20d" exitCode=0 Nov 04 04:02:10.794891 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.794699 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" Nov 04 04:02:10.794891 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.794736 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" event={"ID":"f15a8b2c-4f91-4792-869d-26e339189978","Type":"ContainerDied","Data":"22407c01e6b73c77b5e3523d310725fb5f996d5ddb360a572cda1a492f94f20d"} Nov 04 04:02:10.794891 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.794826 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m" event={"ID":"f15a8b2c-4f91-4792-869d-26e339189978","Type":"ContainerDied","Data":"e0631559dde7614682e079e86346a356cb5ecf776a25ff8a1b12ad481f03cccd"} Nov 04 04:02:10.794891 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.794856 6000 scope.go:122] "RemoveContainer" containerID="22407c01e6b73c77b5e3523d310725fb5f996d5ddb360a572cda1a492f94f20d" Nov 04 04:02:10.796098 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.796074 6000 generic.go:356] "Generic (PLEG): container finished" podID="78a5d7c4-f484-4788-a260-ead9c2ba30da" containerID="2e521d439765f43cc64e1d1576e3eeb17278594e5829e50a9f4865b57eff4de9" exitCode=0 Nov 04 04:02:10.796181 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.796116 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" Nov 04 04:02:10.796181 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.796154 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" event={"ID":"78a5d7c4-f484-4788-a260-ead9c2ba30da","Type":"ContainerDied","Data":"2e521d439765f43cc64e1d1576e3eeb17278594e5829e50a9f4865b57eff4de9"} Nov 04 04:02:10.796249 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.796199 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c" event={"ID":"78a5d7c4-f484-4788-a260-ead9c2ba30da","Type":"ContainerDied","Data":"edfdfc10a69c7c6e84faeccbc6323b5c15cab445449941b0a763ca32eddc190d"} Nov 04 04:02:10.805862 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.805779 6000 scope.go:122] "RemoveContainer" containerID="1052f1e70a12d7d36d17cbef0195b13f379fba4394bbe1e7fde7d4bda4ef6b51" Nov 04 04:02:10.815622 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.815600 6000 scope.go:122] "RemoveContainer" containerID="22407c01e6b73c77b5e3523d310725fb5f996d5ddb360a572cda1a492f94f20d" Nov 04 04:02:10.815997 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:02:10.815975 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22407c01e6b73c77b5e3523d310725fb5f996d5ddb360a572cda1a492f94f20d\": container with ID starting with 22407c01e6b73c77b5e3523d310725fb5f996d5ddb360a572cda1a492f94f20d not found: ID does not exist" containerID="22407c01e6b73c77b5e3523d310725fb5f996d5ddb360a572cda1a492f94f20d" Nov 04 04:02:10.816095 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.816011 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22407c01e6b73c77b5e3523d310725fb5f996d5ddb360a572cda1a492f94f20d"} err="failed to get container status \"22407c01e6b73c77b5e3523d310725fb5f996d5ddb360a572cda1a492f94f20d\": rpc error: code = NotFound desc = could not find container \"22407c01e6b73c77b5e3523d310725fb5f996d5ddb360a572cda1a492f94f20d\": container with ID starting with 22407c01e6b73c77b5e3523d310725fb5f996d5ddb360a572cda1a492f94f20d not found: ID does not exist" Nov 04 04:02:10.816095 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.816037 6000 scope.go:122] "RemoveContainer" containerID="1052f1e70a12d7d36d17cbef0195b13f379fba4394bbe1e7fde7d4bda4ef6b51" Nov 04 04:02:10.816340 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:02:10.816322 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1052f1e70a12d7d36d17cbef0195b13f379fba4394bbe1e7fde7d4bda4ef6b51\": container with ID starting with 1052f1e70a12d7d36d17cbef0195b13f379fba4394bbe1e7fde7d4bda4ef6b51 not found: ID does not exist" containerID="1052f1e70a12d7d36d17cbef0195b13f379fba4394bbe1e7fde7d4bda4ef6b51" Nov 04 04:02:10.816403 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.816351 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1052f1e70a12d7d36d17cbef0195b13f379fba4394bbe1e7fde7d4bda4ef6b51"} err="failed to get container status \"1052f1e70a12d7d36d17cbef0195b13f379fba4394bbe1e7fde7d4bda4ef6b51\": rpc error: code = NotFound desc = could not find container \"1052f1e70a12d7d36d17cbef0195b13f379fba4394bbe1e7fde7d4bda4ef6b51\": container with ID starting with 1052f1e70a12d7d36d17cbef0195b13f379fba4394bbe1e7fde7d4bda4ef6b51 not found: ID does not exist" Nov 04 04:02:10.816403 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.816372 6000 scope.go:122] "RemoveContainer" containerID="2e521d439765f43cc64e1d1576e3eeb17278594e5829e50a9f4865b57eff4de9" Nov 04 04:02:10.825625 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.825599 6000 scope.go:122] "RemoveContainer" containerID="2e521d439765f43cc64e1d1576e3eeb17278594e5829e50a9f4865b57eff4de9" Nov 04 04:02:10.826011 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:02:10.825990 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e521d439765f43cc64e1d1576e3eeb17278594e5829e50a9f4865b57eff4de9\": container with ID starting with 2e521d439765f43cc64e1d1576e3eeb17278594e5829e50a9f4865b57eff4de9 not found: ID does not exist" containerID="2e521d439765f43cc64e1d1576e3eeb17278594e5829e50a9f4865b57eff4de9" Nov 04 04:02:10.826080 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.826024 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e521d439765f43cc64e1d1576e3eeb17278594e5829e50a9f4865b57eff4de9"} err="failed to get container status \"2e521d439765f43cc64e1d1576e3eeb17278594e5829e50a9f4865b57eff4de9\": rpc error: code = NotFound desc = could not find container \"2e521d439765f43cc64e1d1576e3eeb17278594e5829e50a9f4865b57eff4de9\": container with ID starting with 2e521d439765f43cc64e1d1576e3eeb17278594e5829e50a9f4865b57eff4de9 not found: ID does not exist" Nov 04 04:02:10.838454 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.838416 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m"] Nov 04 04:02:10.871489 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.871455 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-f866m"] Nov 04 04:02:10.911657 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.911606 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c"] Nov 04 04:02:10.914021 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:10.913984 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-hvn5c"] Nov 04 04:02:12.081061 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.081026 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="78a5d7c4-f484-4788-a260-ead9c2ba30da" path="/var/lib/kubelet/pods/78a5d7c4-f484-4788-a260-ead9c2ba30da/volumes" Nov 04 04:02:12.081516 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.081463 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f15a8b2c-4f91-4792-869d-26e339189978" path="/var/lib/kubelet/pods/f15a8b2c-4f91-4792-869d-26e339189978/volumes" Nov 04 04:02:12.813444 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.813007 6000 generic.go:356] "Generic (PLEG): container finished" podID="4afd26c6-87c7-4e7e-a315-5e4106ce8a21" containerID="a7bc59fec6b43e18b21b77cb26a6de51a2c010128600ece5a62aa2373eb0db96" exitCode=0 Nov 04 04:02:12.813444 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.813060 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" event={"ID":"4afd26c6-87c7-4e7e-a315-5e4106ce8a21","Type":"ContainerDied","Data":"a7bc59fec6b43e18b21b77cb26a6de51a2c010128600ece5a62aa2373eb0db96"} Nov 04 04:02:12.815292 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.815266 6000 generic.go:356] "Generic (PLEG): container finished" podID="a8354d1d-fe79-42ca-b453-e8b315d4b9a8" containerID="7400917a30dea2e8b404edff1f038799c72a46b0174da8545b6db2750a64f8a5" exitCode=0 Nov 04 04:02:12.815449 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.815347 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" event={"ID":"a8354d1d-fe79-42ca-b453-e8b315d4b9a8","Type":"ContainerDied","Data":"7400917a30dea2e8b404edff1f038799c72a46b0174da8545b6db2750a64f8a5"} Nov 04 04:02:12.817209 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.817188 6000 generic.go:356] "Generic (PLEG): container finished" podID="c1aa05bb-e6a2-49bd-96b3-ec4122ba1391" containerID="c63cebc4a4651159dc6c37458c190548483d0cb937e7b75ecf12a273cbf361fe" exitCode=0 Nov 04 04:02:12.817364 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.817268 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" event={"ID":"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391","Type":"ContainerDied","Data":"c63cebc4a4651159dc6c37458c190548483d0cb937e7b75ecf12a273cbf361fe"} Nov 04 04:02:12.868289 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.867444 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:02:12.930957 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.930927 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:02:12.943941 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.943914 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:02:12.962276 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.962245 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-data\") pod \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " Nov 04 04:02:12.962276 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.962279 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-data\") pod \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " Nov 04 04:02:12.962538 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.962305 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-credential-socket\") pod \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " Nov 04 04:02:12.962538 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.962343 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istiod-ca-cert\") pod \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " Nov 04 04:02:12.962538 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.962388 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-token\" (UniqueName: \"kubernetes.io/projected/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-token\") pod \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " Nov 04 04:02:12.962538 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.962416 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-workload-certs\") pod \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " Nov 04 04:02:12.962777 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.962580 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-data" pod "c1aa05bb-e6a2-49bd-96b3-ec4122ba1391" (UID: "c1aa05bb-e6a2-49bd-96b3-ec4122ba1391"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:12.962777 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.962655 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-podinfo\") pod \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " Nov 04 04:02:12.962777 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.962665 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-credential-socket" pod "c1aa05bb-e6a2-49bd-96b3-ec4122ba1391" (UID: "c1aa05bb-e6a2-49bd-96b3-ec4122ba1391"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:12.962777 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.962691 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-workload-certs" pod "4afd26c6-87c7-4e7e-a315-5e4106ce8a21" (UID: "4afd26c6-87c7-4e7e-a315-5e4106ce8a21"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:12.962777 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.962703 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istiod-ca-cert\") pod \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " Nov 04 04:02:12.962777 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.962747 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-kube-api-access-s5chw\" (UniqueName: \"kubernetes.io/projected/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-kube-api-access-s5chw\") pod \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " Nov 04 04:02:12.963091 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.962798 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-podinfo\") pod \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " Nov 04 04:02:12.963091 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.962846 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-kube-api-access-cl6sx\" (UniqueName: \"kubernetes.io/projected/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-kube-api-access-cl6sx\") pod \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " Nov 04 04:02:12.963091 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.962892 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-envoy\") pod \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " Nov 04 04:02:12.963091 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.962972 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-credential-socket\") pod \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " Nov 04 04:02:12.963091 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.963025 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-workload-socket\") pod \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " Nov 04 04:02:12.963091 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.963067 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-workload-socket\") pod \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " Nov 04 04:02:12.963091 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.963071 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istiod-ca-cert" pod "c1aa05bb-e6a2-49bd-96b3-ec4122ba1391" (UID: "c1aa05bb-e6a2-49bd-96b3-ec4122ba1391"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:02:12.963448 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.963099 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-envoy\") pod \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " Nov 04 04:02:12.963448 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.963126 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-token\" (UniqueName: \"kubernetes.io/projected/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-token\") pod \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\" (UID: \"4afd26c6-87c7-4e7e-a315-5e4106ce8a21\") " Nov 04 04:02:12.963448 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.963137 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-data" pod "4afd26c6-87c7-4e7e-a315-5e4106ce8a21" (UID: "4afd26c6-87c7-4e7e-a315-5e4106ce8a21"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:12.963448 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.963158 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-workload-certs\") pod \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\" (UID: \"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391\") " Nov 04 04:02:12.963448 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.963307 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-workload-socket" pod "4afd26c6-87c7-4e7e-a315-5e4106ce8a21" (UID: "4afd26c6-87c7-4e7e-a315-5e4106ce8a21"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:12.963701 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.963584 6000 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-workload-socket\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:12.963701 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.963606 6000 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-data\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:12.963701 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.963621 6000 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-data\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:12.963701 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.963634 6000 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-credential-socket\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:12.963701 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.963651 6000 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-workload-certs\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:12.963701 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.963666 6000 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istiod-ca-cert\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:12.964051 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.963804 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istiod-ca-cert" pod "4afd26c6-87c7-4e7e-a315-5e4106ce8a21" (UID: "4afd26c6-87c7-4e7e-a315-5e4106ce8a21"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:02:12.964111 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.964063 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-workload-certs" pod "c1aa05bb-e6a2-49bd-96b3-ec4122ba1391" (UID: "c1aa05bb-e6a2-49bd-96b3-ec4122ba1391"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:12.964576 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.964548 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-credential-socket" pod "4afd26c6-87c7-4e7e-a315-5e4106ce8a21" (UID: "4afd26c6-87c7-4e7e-a315-5e4106ce8a21"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:12.964689 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.964623 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-workload-socket" pod "c1aa05bb-e6a2-49bd-96b3-ec4122ba1391" (UID: "c1aa05bb-e6a2-49bd-96b3-ec4122ba1391"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:12.965479 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.965446 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-podinfo" pod "c1aa05bb-e6a2-49bd-96b3-ec4122ba1391" (UID: "c1aa05bb-e6a2-49bd-96b3-ec4122ba1391"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 04:02:12.965643 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.965615 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-kube-api-access-s5chw" pod "4afd26c6-87c7-4e7e-a315-5e4106ce8a21" (UID: "4afd26c6-87c7-4e7e-a315-5e4106ce8a21"). InnerVolumeSpecName "kube-api-access-s5chw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:02:12.966425 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.966398 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-envoy" pod "4afd26c6-87c7-4e7e-a315-5e4106ce8a21" (UID: "4afd26c6-87c7-4e7e-a315-5e4106ce8a21"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:12.968144 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.968084 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-envoy" pod "c1aa05bb-e6a2-49bd-96b3-ec4122ba1391" (UID: "c1aa05bb-e6a2-49bd-96b3-ec4122ba1391"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:12.968144 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.968085 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-kube-api-access-cl6sx" pod "c1aa05bb-e6a2-49bd-96b3-ec4122ba1391" (UID: "c1aa05bb-e6a2-49bd-96b3-ec4122ba1391"). InnerVolumeSpecName "kube-api-access-cl6sx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:02:12.969119 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.969089 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-podinfo" pod "4afd26c6-87c7-4e7e-a315-5e4106ce8a21" (UID: "4afd26c6-87c7-4e7e-a315-5e4106ce8a21"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 04:02:12.969119 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.969098 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-token" pod "c1aa05bb-e6a2-49bd-96b3-ec4122ba1391" (UID: "c1aa05bb-e6a2-49bd-96b3-ec4122ba1391"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:02:12.969682 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:12.969658 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-token" pod "4afd26c6-87c7-4e7e-a315-5e4106ce8a21" (UID: "4afd26c6-87c7-4e7e-a315-5e4106ce8a21"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:02:13.064722 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.064691 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-podinfo\") pod \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " Nov 04 04:02:13.064938 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.064737 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-kube-api-access-lvjdr\" (UniqueName: \"kubernetes.io/projected/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-kube-api-access-lvjdr\") pod \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " Nov 04 04:02:13.064938 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.064798 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-data\") pod \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " Nov 04 04:02:13.064938 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.064854 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-workload-certs\") pod \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " Nov 04 04:02:13.064938 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.064919 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-credential-socket\") pod \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " Nov 04 04:02:13.065149 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.064961 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-token\" (UniqueName: \"kubernetes.io/projected/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-token\") pod \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " Nov 04 04:02:13.065149 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.065020 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istiod-ca-cert\") pod \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " Nov 04 04:02:13.065149 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.065052 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-workload-socket\") pod \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " Nov 04 04:02:13.065149 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.065123 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-envoy\") pod \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\" (UID: \"a8354d1d-fe79-42ca-b453-e8b315d4b9a8\") " Nov 04 04:02:13.065343 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.065176 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-data" pod "a8354d1d-fe79-42ca-b453-e8b315d4b9a8" (UID: "a8354d1d-fe79-42ca-b453-e8b315d4b9a8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:13.065343 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.065257 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-workload-certs" pod "a8354d1d-fe79-42ca-b453-e8b315d4b9a8" (UID: "a8354d1d-fe79-42ca-b453-e8b315d4b9a8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:13.065529 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.065507 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cl6sx\" (UniqueName: \"kubernetes.io/projected/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-kube-api-access-cl6sx\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:13.065618 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.065536 6000 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-envoy\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:13.065618 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.065559 6000 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-credential-socket\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:13.065618 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.065575 6000 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-data\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:13.065618 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.065588 6000 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-workload-socket\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:13.065618 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.065602 6000 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-envoy\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:13.065618 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.065615 6000 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-token\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:13.066220 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.065628 6000 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-workload-certs\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:13.066220 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.065642 6000 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-workload-certs\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:13.066220 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.065656 6000 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istiod-ca-cert\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:13.066220 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.065669 6000 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-token\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:13.066220 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.065682 6000 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-istio-podinfo\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:13.066220 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.065696 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5chw\" (UniqueName: \"kubernetes.io/projected/4afd26c6-87c7-4e7e-a315-5e4106ce8a21-kube-api-access-s5chw\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:13.066220 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.065710 6000 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391-istio-podinfo\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:13.066220 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.065552 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istiod-ca-cert" pod "a8354d1d-fe79-42ca-b453-e8b315d4b9a8" (UID: "a8354d1d-fe79-42ca-b453-e8b315d4b9a8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:02:13.066220 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.065932 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-workload-socket" pod "a8354d1d-fe79-42ca-b453-e8b315d4b9a8" (UID: "a8354d1d-fe79-42ca-b453-e8b315d4b9a8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:13.066220 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.065966 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-credential-socket" pod "a8354d1d-fe79-42ca-b453-e8b315d4b9a8" (UID: "a8354d1d-fe79-42ca-b453-e8b315d4b9a8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:13.067303 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.067272 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-token" pod "a8354d1d-fe79-42ca-b453-e8b315d4b9a8" (UID: "a8354d1d-fe79-42ca-b453-e8b315d4b9a8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:02:13.067400 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.067346 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-envoy" pod "a8354d1d-fe79-42ca-b453-e8b315d4b9a8" (UID: "a8354d1d-fe79-42ca-b453-e8b315d4b9a8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:13.067443 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.067398 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-podinfo" pod "a8354d1d-fe79-42ca-b453-e8b315d4b9a8" (UID: "a8354d1d-fe79-42ca-b453-e8b315d4b9a8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 04:02:13.067539 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.067515 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-kube-api-access-lvjdr" pod "a8354d1d-fe79-42ca-b453-e8b315d4b9a8" (UID: "a8354d1d-fe79-42ca-b453-e8b315d4b9a8"). InnerVolumeSpecName "kube-api-access-lvjdr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:02:13.166633 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.166548 6000 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-credential-socket\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:13.166633 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.166583 6000 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-token\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:13.166633 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.166594 6000 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istiod-ca-cert\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:13.166633 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.166603 6000 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-workload-socket\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:13.166633 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.166613 6000 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-envoy\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:13.166633 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.166622 6000 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-istio-podinfo\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:13.166633 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.166631 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvjdr\" (UniqueName: \"kubernetes.io/projected/a8354d1d-fe79-42ca-b453-e8b315d4b9a8-kube-api-access-lvjdr\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:13.824259 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.824222 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" Nov 04 04:02:13.824460 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.824221 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s" event={"ID":"4afd26c6-87c7-4e7e-a315-5e4106ce8a21","Type":"ContainerDied","Data":"e092052b57012ed2d8d76062b4dc4a2a1156222a79b287536ce971025e851be7"} Nov 04 04:02:13.824460 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.824373 6000 scope.go:122] "RemoveContainer" containerID="a7bc59fec6b43e18b21b77cb26a6de51a2c010128600ece5a62aa2373eb0db96" Nov 04 04:02:13.829184 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.829056 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" event={"ID":"a8354d1d-fe79-42ca-b453-e8b315d4b9a8","Type":"ContainerDied","Data":"b438ca74e59b90e51bcd28ae1893c7c177eaaeb5048491538b64e8f2aa3e24a9"} Nov 04 04:02:13.829288 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.829273 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv" Nov 04 04:02:13.837181 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.837145 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" event={"ID":"c1aa05bb-e6a2-49bd-96b3-ec4122ba1391","Type":"ContainerDied","Data":"5315afd663bce289671d41a0cf61c198906c535a543bb442991e8f7a45527661"} Nov 04 04:02:13.837367 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.837301 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442" Nov 04 04:02:13.850550 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.850527 6000 scope.go:122] "RemoveContainer" containerID="7400917a30dea2e8b404edff1f038799c72a46b0174da8545b6db2750a64f8a5" Nov 04 04:02:13.864285 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.864256 6000 scope.go:122] "RemoveContainer" containerID="c63cebc4a4651159dc6c37458c190548483d0cb937e7b75ecf12a273cbf361fe" Nov 04 04:02:13.877238 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.877150 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s"] Nov 04 04:02:13.883767 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.883721 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-klv9s"] Nov 04 04:02:13.901428 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.901391 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv"] Nov 04 04:02:13.916898 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.916863 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-w49zv"] Nov 04 04:02:13.928745 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.928712 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442"] Nov 04 04:02:13.931635 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:13.931601 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-6h442"] Nov 04 04:02:14.084169 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:14.081730 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4afd26c6-87c7-4e7e-a315-5e4106ce8a21" path="/var/lib/kubelet/pods/4afd26c6-87c7-4e7e-a315-5e4106ce8a21/volumes" Nov 04 04:02:14.084169 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:14.082415 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a8354d1d-fe79-42ca-b453-e8b315d4b9a8" path="/var/lib/kubelet/pods/a8354d1d-fe79-42ca-b453-e8b315d4b9a8/volumes" Nov 04 04:02:14.084169 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:14.082941 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c1aa05bb-e6a2-49bd-96b3-ec4122ba1391" path="/var/lib/kubelet/pods/c1aa05bb-e6a2-49bd-96b3-ec4122ba1391/volumes" Nov 04 04:02:15.824266 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:15.824212 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd"] Nov 04 04:02:15.824787 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:15.824547 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" podUID="2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd" containerName="istio-proxy" containerID="cri-o://c9e03e31be5c0d4e6fd3e9a916023c401f51584ffc1c1b749006e764f48adfa0" gracePeriod=30 Nov 04 04:02:20.748370 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:02:20.748322 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 04:02:21.073004 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.072979 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:02:21.142594 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.142555 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istiod-ca-cert\") pod \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " Nov 04 04:02:21.142796 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.142650 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-credential-socket\") pod \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " Nov 04 04:02:21.142796 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.142688 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-workload-socket\") pod \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " Nov 04 04:02:21.142796 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.142728 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-workload-certs\") pod \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " Nov 04 04:02:21.142975 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.142798 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-token\" (UniqueName: \"kubernetes.io/projected/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-token\") pod \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " Nov 04 04:02:21.142975 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.142861 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-podinfo\") pod \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " Nov 04 04:02:21.142975 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.142925 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-data\") pod \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " Nov 04 04:02:21.142975 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.142958 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-envoy\") pod \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " Nov 04 04:02:21.143183 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.142989 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-kube-api-access-hfmcg\" (UniqueName: \"kubernetes.io/projected/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-kube-api-access-hfmcg\") pod \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\" (UID: \"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd\") " Nov 04 04:02:21.143183 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.142995 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istiod-ca-cert" pod "2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd" (UID: "2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:02:21.143183 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.143097 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-workload-certs" pod "2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd" (UID: "2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:21.143183 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.142995 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-workload-socket" pod "2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd" (UID: "2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:21.143183 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.143014 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-credential-socket" pod "2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd" (UID: "2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:21.143344 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.143199 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-data" pod "2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd" (UID: "2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:21.143811 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.143698 6000 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-data\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:21.143811 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.143724 6000 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istiod-ca-cert\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:21.143811 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.143740 6000 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-credential-socket\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:21.143811 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.143768 6000 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-workload-socket\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:21.143811 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.143784 6000 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-workload-certs\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:21.145186 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.145158 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-podinfo" pod "2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd" (UID: "2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 04:02:21.145353 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.145328 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-envoy" pod "2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd" (UID: "2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:02:21.145559 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.145535 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-token" pod "2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd" (UID: "2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:02:21.145618 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.145548 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-kube-api-access-hfmcg" pod "2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd" (UID: "2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd"). InnerVolumeSpecName "kube-api-access-hfmcg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:02:21.245350 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.245305 6000 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-token\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:21.245350 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.245342 6000 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-podinfo\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:21.245350 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.245353 6000 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-istio-envoy\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:21.245598 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.245362 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfmcg\" (UniqueName: \"kubernetes.io/projected/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd-kube-api-access-hfmcg\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:02:21.885290 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.885253 6000 generic.go:356] "Generic (PLEG): container finished" podID="2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd" containerID="c9e03e31be5c0d4e6fd3e9a916023c401f51584ffc1c1b749006e764f48adfa0" exitCode=0 Nov 04 04:02:21.885290 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.885291 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" event={"ID":"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd","Type":"ContainerDied","Data":"c9e03e31be5c0d4e6fd3e9a916023c401f51584ffc1c1b749006e764f48adfa0"} Nov 04 04:02:21.885900 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.885304 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" Nov 04 04:02:21.885900 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.885334 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd" event={"ID":"2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd","Type":"ContainerDied","Data":"e6e6865eed807a04dc9f6d07cdc50f671f121ec42dbcb782e1ea75d12992de90"} Nov 04 04:02:21.885900 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.885355 6000 scope.go:122] "RemoveContainer" containerID="c9e03e31be5c0d4e6fd3e9a916023c401f51584ffc1c1b749006e764f48adfa0" Nov 04 04:02:21.895424 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.895401 6000 scope.go:122] "RemoveContainer" containerID="c9e03e31be5c0d4e6fd3e9a916023c401f51584ffc1c1b749006e764f48adfa0" Nov 04 04:02:21.895732 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:02:21.895711 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9e03e31be5c0d4e6fd3e9a916023c401f51584ffc1c1b749006e764f48adfa0\": container with ID starting with c9e03e31be5c0d4e6fd3e9a916023c401f51584ffc1c1b749006e764f48adfa0 not found: ID does not exist" containerID="c9e03e31be5c0d4e6fd3e9a916023c401f51584ffc1c1b749006e764f48adfa0" Nov 04 04:02:21.895817 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.895740 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9e03e31be5c0d4e6fd3e9a916023c401f51584ffc1c1b749006e764f48adfa0"} err="failed to get container status \"c9e03e31be5c0d4e6fd3e9a916023c401f51584ffc1c1b749006e764f48adfa0\": rpc error: code = NotFound desc = could not find container \"c9e03e31be5c0d4e6fd3e9a916023c401f51584ffc1c1b749006e764f48adfa0\": container with ID starting with c9e03e31be5c0d4e6fd3e9a916023c401f51584ffc1c1b749006e764f48adfa0 not found: ID does not exist" Nov 04 04:02:21.908878 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.908848 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd"] Nov 04 04:02:21.914244 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:21.914216 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-zjwdd"] Nov 04 04:02:22.080293 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:22.080259 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd" path="/var/lib/kubelet/pods/2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd/volumes" Nov 04 04:02:29.475517 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:29.475467 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-5xmx7_6da524eb-f360-4de3-979e-d577dfae526d/manager/0.log" Nov 04 04:02:30.326669 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:30.326625 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-r66wn_0fa75f01-dc06-4030-a496-6a354acf20c2/proxy/0.log" Nov 04 04:02:33.748605 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:33.748572 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-2wcfs_dc3db230-5977-4b27-9938-3d5f62ac53ad/postgres/0.log" Nov 04 04:02:35.755170 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:35.755133 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-2d8lw_b279d26f-2d92-4082-8edb-d1c2a26022aa/manager/0.log" Nov 04 04:02:35.930721 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:35.930688 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-vp5lg_f59b063c-f397-4370-ac72-84335c8115b4/manager/0.log" Nov 04 04:02:36.428106 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:36.428068 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-2jdbl_75ff9606-9236-43ed-b04f-d91f35ed4ea7/manager/0.log" Nov 04 04:02:36.663435 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:36.663403 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-zf6m4_343c0b0b-406a-4c05-b4dd-5b908f8c8c32/limitador/0.log" Nov 04 04:02:36.841496 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:36.841456 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-p9sng_e84f739c-2745-44c9-ab3c-d6516ab6855a/manager/0.log" Nov 04 04:02:37.762949 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:37.762909 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-597d745469j8tv2_e8873cfa-cbb1-4b2b-aa61-fca152bee660/istio-proxy/0.log" Nov 04 04:02:39.047128 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:39.047076 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-gnswf_01a09535-2de8-4f06-846b-556b9db5e38b/istio-proxy/0.log" Nov 04 04:02:39.214567 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:39.214536 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-kkgr2_ee7b70f9-1025-4144-b882-fb75f1159c3e/istio-proxy/0.log" Nov 04 04:02:39.588430 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:39.588401 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-skgsr_fa196927-03ec-4015-8172-44098857d262/istio-proxy/1.log" Nov 04 04:02:41.234438 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:41.234411 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-skgsr_fa196927-03ec-4015-8172-44098857d262/istio-proxy/0.log" Nov 04 04:02:50.748904 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:02:50.748866 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 04:02:53.510797 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:53.510742 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-5xmx7_6da524eb-f360-4de3-979e-d577dfae526d/manager/0.log" Nov 04 04:02:54.294625 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:54.294594 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-r66wn_0fa75f01-dc06-4030-a496-6a354acf20c2/proxy/0.log" Nov 04 04:02:57.314022 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:57.313993 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-2wcfs_dc3db230-5977-4b27-9938-3d5f62ac53ad/postgres/0.log" Nov 04 04:02:58.572888 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:58.572853 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-2d8lw_b279d26f-2d92-4082-8edb-d1c2a26022aa/manager/0.log" Nov 04 04:02:58.744793 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:58.744735 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-vp5lg_f59b063c-f397-4370-ac72-84335c8115b4/manager/0.log" Nov 04 04:02:59.099382 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:59.099346 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-2jdbl_75ff9606-9236-43ed-b04f-d91f35ed4ea7/manager/0.log" Nov 04 04:02:59.330395 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:59.330363 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-zf6m4_343c0b0b-406a-4c05-b4dd-5b908f8c8c32/limitador/0.log" Nov 04 04:02:59.507420 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:02:59.507335 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-p9sng_e84f739c-2745-44c9-ab3c-d6516ab6855a/manager/0.log" Nov 04 04:03:00.045116 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:03:00.045088 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-597d745469j8tv2_e8873cfa-cbb1-4b2b-aa61-fca152bee660/istio-proxy/0.log" Nov 04 04:03:01.321731 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:03:01.321699 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-gnswf_01a09535-2de8-4f06-846b-556b9db5e38b/istio-proxy/0.log" Nov 04 04:03:01.503615 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:03:01.503577 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-kkgr2_ee7b70f9-1025-4144-b882-fb75f1159c3e/istio-proxy/0.log" Nov 04 04:03:01.857114 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:03:01.857079 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-skgsr_fa196927-03ec-4015-8172-44098857d262/istio-proxy/1.log" Nov 04 04:03:20.747150 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:03:20.747106 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 04:03:50.747478 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:03:50.747389 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 04:04:00.146997 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.146961 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-26w7b/must-gather-ljgbm"] Nov 04 04:04:00.147424 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147323 6000 cpu_manager.go:414] "containerMap: removing container" podUID="31648502-f58d-43a9-a746-bcfb74eb70f7" containerName="payload-processing" Nov 04 04:04:00.147424 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147334 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31648502-f58d-43a9-a746-bcfb74eb70f7" containerName="payload-processing" Nov 04 04:04:00.147424 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147349 6000 cpu_manager.go:414] "containerMap: removing container" podUID="f15a8b2c-4f91-4792-869d-26e339189978" containerName="istio-proxy" Nov 04 04:04:00.147424 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147354 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f15a8b2c-4f91-4792-869d-26e339189978" containerName="istio-proxy" Nov 04 04:04:00.147424 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147362 6000 cpu_manager.go:414] "containerMap: removing container" podUID="97aca100-541e-48ff-9b0e-5d4853b700ff" containerName="maas-api" Nov 04 04:04:00.147424 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147368 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97aca100-541e-48ff-9b0e-5d4853b700ff" containerName="maas-api" Nov 04 04:04:00.147424 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147374 6000 cpu_manager.go:414] "containerMap: removing container" podUID="785d143f-077d-4ebd-8b1f-1cd3c67aac47" containerName="main" Nov 04 04:04:00.147424 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147379 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="785d143f-077d-4ebd-8b1f-1cd3c67aac47" containerName="main" Nov 04 04:04:00.147424 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147394 6000 cpu_manager.go:414] "containerMap: removing container" podUID="4afd26c6-87c7-4e7e-a315-5e4106ce8a21" containerName="istio-proxy" Nov 04 04:04:00.147424 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147399 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4afd26c6-87c7-4e7e-a315-5e4106ce8a21" containerName="istio-proxy" Nov 04 04:04:00.147424 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147407 6000 cpu_manager.go:414] "containerMap: removing container" podUID="f15a8b2c-4f91-4792-869d-26e339189978" containerName="istio-proxy" Nov 04 04:04:00.147424 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147412 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f15a8b2c-4f91-4792-869d-26e339189978" containerName="istio-proxy" Nov 04 04:04:00.147424 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147419 6000 cpu_manager.go:414] "containerMap: removing container" podUID="78a5d7c4-f484-4788-a260-ead9c2ba30da" containerName="istio-proxy" Nov 04 04:04:00.147424 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147423 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78a5d7c4-f484-4788-a260-ead9c2ba30da" containerName="istio-proxy" Nov 04 04:04:00.147943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147441 6000 cpu_manager.go:414] "containerMap: removing container" podUID="eef5b758-9117-4d9a-901c-29f03be11804" containerName="main" Nov 04 04:04:00.147943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147445 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eef5b758-9117-4d9a-901c-29f03be11804" containerName="main" Nov 04 04:04:00.147943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147451 6000 cpu_manager.go:414] "containerMap: removing container" podUID="a8354d1d-fe79-42ca-b453-e8b315d4b9a8" containerName="istio-proxy" Nov 04 04:04:00.147943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147456 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8354d1d-fe79-42ca-b453-e8b315d4b9a8" containerName="istio-proxy" Nov 04 04:04:00.147943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147463 6000 cpu_manager.go:414] "containerMap: removing container" podUID="2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd" containerName="istio-proxy" Nov 04 04:04:00.147943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147469 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd" containerName="istio-proxy" Nov 04 04:04:00.147943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147476 6000 cpu_manager.go:414] "containerMap: removing container" podUID="c1aa05bb-e6a2-49bd-96b3-ec4122ba1391" containerName="istio-proxy" Nov 04 04:04:00.147943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147480 6000 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c1aa05bb-e6a2-49bd-96b3-ec4122ba1391" containerName="istio-proxy" Nov 04 04:04:00.147943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147531 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="97aca100-541e-48ff-9b0e-5d4853b700ff" containerName="maas-api" Nov 04 04:04:00.147943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147539 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="785d143f-077d-4ebd-8b1f-1cd3c67aac47" containerName="main" Nov 04 04:04:00.147943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147544 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="78a5d7c4-f484-4788-a260-ead9c2ba30da" containerName="istio-proxy" Nov 04 04:04:00.147943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147552 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="4afd26c6-87c7-4e7e-a315-5e4106ce8a21" containerName="istio-proxy" Nov 04 04:04:00.147943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147557 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="f15a8b2c-4f91-4792-869d-26e339189978" containerName="istio-proxy" Nov 04 04:04:00.147943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147565 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="eef5b758-9117-4d9a-901c-29f03be11804" containerName="main" Nov 04 04:04:00.147943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147572 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8354d1d-fe79-42ca-b453-e8b315d4b9a8" containerName="istio-proxy" Nov 04 04:04:00.147943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147578 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f6ecbe7-6df4-43c3-b913-9dfd6e1f79cd" containerName="istio-proxy" Nov 04 04:04:00.147943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147583 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="c1aa05bb-e6a2-49bd-96b3-ec4122ba1391" containerName="istio-proxy" Nov 04 04:04:00.147943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147588 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="31648502-f58d-43a9-a746-bcfb74eb70f7" containerName="payload-processing" Nov 04 04:04:00.147943 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.147696 6000 memory_manager.go:365] "RemoveStaleState removing state" podUID="f15a8b2c-4f91-4792-869d-26e339189978" containerName="istio-proxy" Nov 04 04:04:00.150813 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.150793 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-26w7b/must-gather-ljgbm" Nov 04 04:04:00.155271 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.155243 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-26w7b\"/\"kube-root-ca.crt\"" Nov 04 04:04:00.156310 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.156288 6000 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-26w7b\"/\"openshift-service-ca.crt\"" Nov 04 04:04:00.156448 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.156328 6000 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-26w7b\"/\"default-dockercfg-4hjnm\"" Nov 04 04:04:00.175088 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.174968 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-26w7b/must-gather-ljgbm"] Nov 04 04:04:00.242481 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.242440 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpjbb\" (UniqueName: \"kubernetes.io/projected/c0ea9048-fb50-4080-9c9f-e04a61a507a7-kube-api-access-dpjbb\") pod \"must-gather-ljgbm\" (UID: \"c0ea9048-fb50-4080-9c9f-e04a61a507a7\") " pod="openshift-must-gather-26w7b/must-gather-ljgbm" Nov 04 04:04:00.242700 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.242620 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c0ea9048-fb50-4080-9c9f-e04a61a507a7-must-gather-output\") pod \"must-gather-ljgbm\" (UID: \"c0ea9048-fb50-4080-9c9f-e04a61a507a7\") " pod="openshift-must-gather-26w7b/must-gather-ljgbm" Nov 04 04:04:00.343861 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.343823 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c0ea9048-fb50-4080-9c9f-e04a61a507a7-must-gather-output\") pod \"must-gather-ljgbm\" (UID: \"c0ea9048-fb50-4080-9c9f-e04a61a507a7\") " pod="openshift-must-gather-26w7b/must-gather-ljgbm" Nov 04 04:04:00.344051 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.343880 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpjbb\" (UniqueName: \"kubernetes.io/projected/c0ea9048-fb50-4080-9c9f-e04a61a507a7-kube-api-access-dpjbb\") pod \"must-gather-ljgbm\" (UID: \"c0ea9048-fb50-4080-9c9f-e04a61a507a7\") " pod="openshift-must-gather-26w7b/must-gather-ljgbm" Nov 04 04:04:00.344251 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.344226 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c0ea9048-fb50-4080-9c9f-e04a61a507a7-must-gather-output\") pod \"must-gather-ljgbm\" (UID: \"c0ea9048-fb50-4080-9c9f-e04a61a507a7\") " pod="openshift-must-gather-26w7b/must-gather-ljgbm" Nov 04 04:04:00.356460 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.356428 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpjbb\" (UniqueName: \"kubernetes.io/projected/c0ea9048-fb50-4080-9c9f-e04a61a507a7-kube-api-access-dpjbb\") pod \"must-gather-ljgbm\" (UID: \"c0ea9048-fb50-4080-9c9f-e04a61a507a7\") " pod="openshift-must-gather-26w7b/must-gather-ljgbm" Nov 04 04:04:00.476355 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.476258 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-26w7b/must-gather-ljgbm" Nov 04 04:04:00.753870 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.753810 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-26w7b/must-gather-ljgbm"] Nov 04 04:04:00.756306 ip-10-0-129-134 kubenswrapper[6000]: W1104 04:04:00.756273 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0ea9048_fb50_4080_9c9f_e04a61a507a7.slice/crio-a978d1249c26d8eb170fe01dd6a8d9214ef7ec2a2ff8dc76c2f939f4e3c7c6cd WatchSource:0}: Error finding container a978d1249c26d8eb170fe01dd6a8d9214ef7ec2a2ff8dc76c2f939f4e3c7c6cd: Status 404 returned error can't find the container with id a978d1249c26d8eb170fe01dd6a8d9214ef7ec2a2ff8dc76c2f939f4e3c7c6cd Nov 04 04:04:00.758356 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:00.758332 6000 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 04:04:01.390431 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:01.390393 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-26w7b/must-gather-ljgbm" event={"ID":"c0ea9048-fb50-4080-9c9f-e04a61a507a7","Type":"ContainerStarted","Data":"a978d1249c26d8eb170fe01dd6a8d9214ef7ec2a2ff8dc76c2f939f4e3c7c6cd"} Nov 04 04:04:02.400239 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:02.400199 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-26w7b/must-gather-ljgbm" event={"ID":"c0ea9048-fb50-4080-9c9f-e04a61a507a7","Type":"ContainerStarted","Data":"8b255204b7127aaa64163b0ec05eb9daecbc2116cb1b1abac3be84e9a7e086b2"} Nov 04 04:04:02.400772 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:02.400251 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-26w7b/must-gather-ljgbm" event={"ID":"c0ea9048-fb50-4080-9c9f-e04a61a507a7","Type":"ContainerStarted","Data":"3d63a035ca23d7dc3f857e18b25c8b65beeff8b2ce74dea9de35fb901ec2e024"} Nov 04 04:04:03.635088 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:03.635057 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-p84rx_06a274ba-93f4-4dbc-ac39-4039fd254e7d/global-pull-secret-syncer/0.log" Nov 04 04:04:03.843560 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:03.843514 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-895vv_dde9ca65-5089-450e-97cd-89bf66adba30/konnectivity-agent/0.log" Nov 04 04:04:03.907271 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:03.907180 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-134.us-east-2.compute.internal_18d40cab8244943c9879207cd43e3bc3/haproxy/0.log" Nov 04 04:04:10.343479 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:10.343349 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-2d8lw_b279d26f-2d92-4082-8edb-d1c2a26022aa/manager/0.log" Nov 04 04:04:10.369564 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:10.369518 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-vp5lg_f59b063c-f397-4370-ac72-84335c8115b4/manager/0.log" Nov 04 04:04:10.724778 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:10.724340 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-2jdbl_75ff9606-9236-43ed-b04f-d91f35ed4ea7/manager/0.log" Nov 04 04:04:10.762897 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:10.762151 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-zf6m4_343c0b0b-406a-4c05-b4dd-5b908f8c8c32/limitador/0.log" Nov 04 04:04:10.820780 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:10.820309 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-p9sng_e84f739c-2745-44c9-ab3c-d6516ab6855a/manager/0.log" Nov 04 04:04:12.637081 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:12.636959 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-26w7b/must-gather-ljgbm" podStartSLOduration=11.671272427 podStartE2EDuration="12.636937503s" totalImagesPullingTime="965.665076ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:04:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:04:02.41992602 +0000 UTC m=+840.458365984" watchObservedRunningTime="2026-11-04 04:04:12.636937503 +0000 UTC m=+850.675377468" Nov 04 04:04:12.637984 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:12.637951 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf"] Nov 04 04:04:12.638267 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:12.638242 6000 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" podUID="01a09535-2de8-4f06-846b-556b9db5e38b" containerName="istio-proxy" containerID="cri-o://ed1fd7d34aed2eb2ad514c40d4ff506bb1a412471859d53e3d8faca54d3e1cb5" gracePeriod=30 Nov 04 04:04:12.722776 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:12.722654 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-f92gv_27390c92-0480-4b55-ba3d-e20f7ada5b83/monitoring-plugin/0.log" Nov 04 04:04:12.985374 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:12.985274 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r7srf_b1091f60-668b-4884-8548-fe4f4c438367/node-exporter/0.log" Nov 04 04:04:13.012065 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.012033 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r7srf_b1091f60-668b-4884-8548-fe4f4c438367/kube-rbac-proxy/0.log" Nov 04 04:04:13.042912 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.042865 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r7srf_b1091f60-668b-4884-8548-fe4f4c438367/init-textfile/0.log" Nov 04 04:04:13.178858 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.178824 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_38fe00d5-85df-48ec-8611-1a7eb735a845/prometheus/0.log" Nov 04 04:04:13.208174 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.208141 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_38fe00d5-85df-48ec-8611-1a7eb735a845/config-reloader/0.log" Nov 04 04:04:13.241256 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.241163 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_38fe00d5-85df-48ec-8611-1a7eb735a845/thanos-sidecar/0.log" Nov 04 04:04:13.276149 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.276076 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_38fe00d5-85df-48ec-8611-1a7eb735a845/kube-rbac-proxy-web/0.log" Nov 04 04:04:13.308530 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.308484 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_38fe00d5-85df-48ec-8611-1a7eb735a845/kube-rbac-proxy/0.log" Nov 04 04:04:13.337613 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.337585 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_38fe00d5-85df-48ec-8611-1a7eb735a845/kube-rbac-proxy-thanos/0.log" Nov 04 04:04:13.367091 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.367058 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_38fe00d5-85df-48ec-8611-1a7eb735a845/init-config-reloader/0.log" Nov 04 04:04:13.401446 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.401413 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-tnr8p_2502af7a-1118-4c5e-84b8-8c97188c09a4/prometheus-operator/0.log" Nov 04 04:04:13.429008 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.428974 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-tnr8p_2502af7a-1118-4c5e-84b8-8c97188c09a4/kube-rbac-proxy/0.log" Nov 04 04:04:13.603690 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.603658 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7976867cc6-jgp9n_8a6918a9-3ff2-46aa-bc91-affd1835403d/thanos-query/0.log" Nov 04 04:04:13.641194 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.641166 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7976867cc6-jgp9n_8a6918a9-3ff2-46aa-bc91-affd1835403d/kube-rbac-proxy-web/0.log" Nov 04 04:04:13.673544 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.673468 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7976867cc6-jgp9n_8a6918a9-3ff2-46aa-bc91-affd1835403d/kube-rbac-proxy/0.log" Nov 04 04:04:13.708081 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.708040 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7976867cc6-jgp9n_8a6918a9-3ff2-46aa-bc91-affd1835403d/prom-label-proxy/0.log" Nov 04 04:04:13.738381 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.738322 6000 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-26w7b/perf-node-gather-daemonset-6pnpf"] Nov 04 04:04:13.745845 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.745806 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-6pnpf" Nov 04 04:04:13.752950 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.752916 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-26w7b/perf-node-gather-daemonset-6pnpf"] Nov 04 04:04:13.753197 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.753029 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7976867cc6-jgp9n_8a6918a9-3ff2-46aa-bc91-affd1835403d/kube-rbac-proxy-rules/0.log" Nov 04 04:04:13.784892 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.784853 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7976867cc6-jgp9n_8a6918a9-3ff2-46aa-bc91-affd1835403d/kube-rbac-proxy-metrics/0.log" Nov 04 04:04:13.829797 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.829277 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/465452c9-7cc6-4a2c-a344-32d03065e95d-sys\") pod \"perf-node-gather-daemonset-6pnpf\" (UID: \"465452c9-7cc6-4a2c-a344-32d03065e95d\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-6pnpf" Nov 04 04:04:13.829797 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.829337 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/465452c9-7cc6-4a2c-a344-32d03065e95d-proc\") pod \"perf-node-gather-daemonset-6pnpf\" (UID: \"465452c9-7cc6-4a2c-a344-32d03065e95d\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-6pnpf" Nov 04 04:04:13.829797 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.829379 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srvml\" (UniqueName: \"kubernetes.io/projected/465452c9-7cc6-4a2c-a344-32d03065e95d-kube-api-access-srvml\") pod \"perf-node-gather-daemonset-6pnpf\" (UID: \"465452c9-7cc6-4a2c-a344-32d03065e95d\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-6pnpf" Nov 04 04:04:13.829797 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.829423 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/465452c9-7cc6-4a2c-a344-32d03065e95d-podres\") pod \"perf-node-gather-daemonset-6pnpf\" (UID: \"465452c9-7cc6-4a2c-a344-32d03065e95d\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-6pnpf" Nov 04 04:04:13.829797 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.829458 6000 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/465452c9-7cc6-4a2c-a344-32d03065e95d-lib-modules\") pod \"perf-node-gather-daemonset-6pnpf\" (UID: \"465452c9-7cc6-4a2c-a344-32d03065e95d\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-6pnpf" Nov 04 04:04:13.930647 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.930561 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/465452c9-7cc6-4a2c-a344-32d03065e95d-sys\") pod \"perf-node-gather-daemonset-6pnpf\" (UID: \"465452c9-7cc6-4a2c-a344-32d03065e95d\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-6pnpf" Nov 04 04:04:13.930861 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.930846 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/465452c9-7cc6-4a2c-a344-32d03065e95d-proc\") pod \"perf-node-gather-daemonset-6pnpf\" (UID: \"465452c9-7cc6-4a2c-a344-32d03065e95d\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-6pnpf" Nov 04 04:04:13.930969 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.930959 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srvml\" (UniqueName: \"kubernetes.io/projected/465452c9-7cc6-4a2c-a344-32d03065e95d-kube-api-access-srvml\") pod \"perf-node-gather-daemonset-6pnpf\" (UID: \"465452c9-7cc6-4a2c-a344-32d03065e95d\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-6pnpf" Nov 04 04:04:13.931063 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.931053 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/465452c9-7cc6-4a2c-a344-32d03065e95d-podres\") pod \"perf-node-gather-daemonset-6pnpf\" (UID: \"465452c9-7cc6-4a2c-a344-32d03065e95d\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-6pnpf" Nov 04 04:04:13.931156 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.931147 6000 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/465452c9-7cc6-4a2c-a344-32d03065e95d-lib-modules\") pod \"perf-node-gather-daemonset-6pnpf\" (UID: \"465452c9-7cc6-4a2c-a344-32d03065e95d\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-6pnpf" Nov 04 04:04:13.931366 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.931346 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/465452c9-7cc6-4a2c-a344-32d03065e95d-lib-modules\") pod \"perf-node-gather-daemonset-6pnpf\" (UID: \"465452c9-7cc6-4a2c-a344-32d03065e95d\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-6pnpf" Nov 04 04:04:13.931486 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.931476 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/465452c9-7cc6-4a2c-a344-32d03065e95d-sys\") pod \"perf-node-gather-daemonset-6pnpf\" (UID: \"465452c9-7cc6-4a2c-a344-32d03065e95d\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-6pnpf" Nov 04 04:04:13.931585 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.931575 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/465452c9-7cc6-4a2c-a344-32d03065e95d-proc\") pod \"perf-node-gather-daemonset-6pnpf\" (UID: \"465452c9-7cc6-4a2c-a344-32d03065e95d\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-6pnpf" Nov 04 04:04:13.931890 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.931864 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/465452c9-7cc6-4a2c-a344-32d03065e95d-podres\") pod \"perf-node-gather-daemonset-6pnpf\" (UID: \"465452c9-7cc6-4a2c-a344-32d03065e95d\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-6pnpf" Nov 04 04:04:13.949812 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:13.949739 6000 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srvml\" (UniqueName: \"kubernetes.io/projected/465452c9-7cc6-4a2c-a344-32d03065e95d-kube-api-access-srvml\") pod \"perf-node-gather-daemonset-6pnpf\" (UID: \"465452c9-7cc6-4a2c-a344-32d03065e95d\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-6pnpf" Nov 04 04:04:14.078234 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:14.077978 6000 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-6pnpf" Nov 04 04:04:14.372848 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:14.372808 6000 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-26w7b/perf-node-gather-daemonset-6pnpf"] Nov 04 04:04:14.377746 ip-10-0-129-134 kubenswrapper[6000]: W1104 04:04:14.377708 6000 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod465452c9_7cc6_4a2c_a344_32d03065e95d.slice/crio-ce8501d5a6d64ed89ed7446b3b0a8e0e6f08cd2aac1443081dbb213799270f4a WatchSource:0}: Error finding container ce8501d5a6d64ed89ed7446b3b0a8e0e6f08cd2aac1443081dbb213799270f4a: Status 404 returned error can't find the container with id ce8501d5a6d64ed89ed7446b3b0a8e0e6f08cd2aac1443081dbb213799270f4a Nov 04 04:04:14.521606 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:14.521557 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-6pnpf" event={"ID":"465452c9-7cc6-4a2c-a344-32d03065e95d","Type":"ContainerStarted","Data":"ce8501d5a6d64ed89ed7446b3b0a8e0e6f08cd2aac1443081dbb213799270f4a"} Nov 04 04:04:15.530077 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:15.530040 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-6pnpf" event={"ID":"465452c9-7cc6-4a2c-a344-32d03065e95d","Type":"ContainerStarted","Data":"e64e2ceca292e34ffe4ab5a1569cefd34f7894dab81f0b7dcbe268f12db768c7"} Nov 04 04:04:15.530492 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:15.530214 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-6pnpf" Nov 04 04:04:15.551170 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:15.551117 6000 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-6pnpf" podStartSLOduration=2.55110333 podStartE2EDuration="2.55110333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 04:04:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:04:15.548928874 +0000 UTC m=+853.587368840" watchObservedRunningTime="2026-11-04 04:04:15.55110333 +0000 UTC m=+853.589543294" Nov 04 04:04:17.942794 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:17.941886 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 04:04:17.991351 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:17.991309 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/01a09535-2de8-4f06-846b-556b9db5e38b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01a09535-2de8-4f06-846b-556b9db5e38b-istiod-ca-cert\") pod \"01a09535-2de8-4f06-846b-556b9db5e38b\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " Nov 04 04:04:17.991532 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:17.991359 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/01a09535-2de8-4f06-846b-556b9db5e38b-istio-token\" (UniqueName: \"kubernetes.io/projected/01a09535-2de8-4f06-846b-556b9db5e38b-istio-token\") pod \"01a09535-2de8-4f06-846b-556b9db5e38b\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " Nov 04 04:04:17.991532 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:17.991513 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/01a09535-2de8-4f06-846b-556b9db5e38b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01a09535-2de8-4f06-846b-556b9db5e38b-istio-podinfo\") pod \"01a09535-2de8-4f06-846b-556b9db5e38b\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " Nov 04 04:04:17.991661 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:17.991564 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-credential-socket\") pod \"01a09535-2de8-4f06-846b-556b9db5e38b\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " Nov 04 04:04:17.991661 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:17.991604 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-workload-certs\") pod \"01a09535-2de8-4f06-846b-556b9db5e38b\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " Nov 04 04:04:17.991776 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:17.991672 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-workload-socket\") pod \"01a09535-2de8-4f06-846b-556b9db5e38b\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " Nov 04 04:04:17.991776 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:17.991710 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/01a09535-2de8-4f06-846b-556b9db5e38b-kube-api-access-cbpqm\" (UniqueName: \"kubernetes.io/projected/01a09535-2de8-4f06-846b-556b9db5e38b-kube-api-access-cbpqm\") pod \"01a09535-2de8-4f06-846b-556b9db5e38b\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " Nov 04 04:04:17.991776 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:17.991729 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01a09535-2de8-4f06-846b-556b9db5e38b-istiod-ca-cert" pod "01a09535-2de8-4f06-846b-556b9db5e38b" (UID: "01a09535-2de8-4f06-846b-556b9db5e38b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:04:17.991928 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:17.991751 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-istio-data\") pod \"01a09535-2de8-4f06-846b-556b9db5e38b\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " Nov 04 04:04:17.991928 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:17.991816 6000 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-istio-envoy\") pod \"01a09535-2de8-4f06-846b-556b9db5e38b\" (UID: \"01a09535-2de8-4f06-846b-556b9db5e38b\") " Nov 04 04:04:17.992023 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:17.991958 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-workload-certs" pod "01a09535-2de8-4f06-846b-556b9db5e38b" (UID: "01a09535-2de8-4f06-846b-556b9db5e38b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:04:17.992023 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:17.992011 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-credential-socket" pod "01a09535-2de8-4f06-846b-556b9db5e38b" (UID: "01a09535-2de8-4f06-846b-556b9db5e38b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:04:17.992236 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:17.992165 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-istio-data" pod "01a09535-2de8-4f06-846b-556b9db5e38b" (UID: "01a09535-2de8-4f06-846b-556b9db5e38b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:04:17.992236 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:17.992183 6000 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01a09535-2de8-4f06-846b-556b9db5e38b-istiod-ca-cert\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:04:17.992236 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:17.992208 6000 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-credential-socket\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:04:17.992236 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:17.992222 6000 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-workload-certs\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:04:17.992463 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:17.992439 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-workload-socket" pod "01a09535-2de8-4f06-846b-556b9db5e38b" (UID: "01a09535-2de8-4f06-846b-556b9db5e38b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:04:17.994792 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:17.994180 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/01a09535-2de8-4f06-846b-556b9db5e38b-istio-podinfo" pod "01a09535-2de8-4f06-846b-556b9db5e38b" (UID: "01a09535-2de8-4f06-846b-556b9db5e38b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 04:04:17.994792 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:17.994309 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01a09535-2de8-4f06-846b-556b9db5e38b-istio-token" pod "01a09535-2de8-4f06-846b-556b9db5e38b" (UID: "01a09535-2de8-4f06-846b-556b9db5e38b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:04:17.994792 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:17.994556 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01a09535-2de8-4f06-846b-556b9db5e38b-kube-api-access-cbpqm" pod "01a09535-2de8-4f06-846b-556b9db5e38b" (UID: "01a09535-2de8-4f06-846b-556b9db5e38b"). InnerVolumeSpecName "kube-api-access-cbpqm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:04:17.995036 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:17.994866 6000 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-istio-envoy" pod "01a09535-2de8-4f06-846b-556b9db5e38b" (UID: "01a09535-2de8-4f06-846b-556b9db5e38b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:04:18.094565 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:18.094531 6000 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01a09535-2de8-4f06-846b-556b9db5e38b-istio-podinfo\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:04:18.094565 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:18.094564 6000 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-workload-socket\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:04:18.094747 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:18.094575 6000 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbpqm\" (UniqueName: \"kubernetes.io/projected/01a09535-2de8-4f06-846b-556b9db5e38b-kube-api-access-cbpqm\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:04:18.094747 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:18.094585 6000 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-istio-data\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:04:18.094747 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:18.094593 6000 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01a09535-2de8-4f06-846b-556b9db5e38b-istio-envoy\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:04:18.094747 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:18.094602 6000 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01a09535-2de8-4f06-846b-556b9db5e38b-istio-token\") on node \"ip-10-0-129-134.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 04:04:18.138144 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:18.138095 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vcxpf_94c06e7a-0510-4787-8b60-45b185300091/dns/0.log" Nov 04 04:04:18.165683 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:18.165596 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vcxpf_94c06e7a-0510-4787-8b60-45b185300091/kube-rbac-proxy/0.log" Nov 04 04:04:18.259403 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:18.259375 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dxmg7_2b70504f-4169-44b2-8b4e-979a068f070d/dns-node-resolver/0.log" Nov 04 04:04:18.571347 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:18.571305 6000 generic.go:356] "Generic (PLEG): container finished" podID="01a09535-2de8-4f06-846b-556b9db5e38b" containerID="ed1fd7d34aed2eb2ad514c40d4ff506bb1a412471859d53e3d8faca54d3e1cb5" exitCode=0 Nov 04 04:04:18.571532 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:18.571373 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" event={"ID":"01a09535-2de8-4f06-846b-556b9db5e38b","Type":"ContainerDied","Data":"ed1fd7d34aed2eb2ad514c40d4ff506bb1a412471859d53e3d8faca54d3e1cb5"} Nov 04 04:04:18.571532 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:18.571390 6000 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" Nov 04 04:04:18.571532 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:18.571431 6000 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf" event={"ID":"01a09535-2de8-4f06-846b-556b9db5e38b","Type":"ContainerDied","Data":"4335ff87952ef9471b74e35d39057ac84a3949f912b8204b4c12bfe3c22af5df"} Nov 04 04:04:18.571532 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:18.571464 6000 scope.go:122] "RemoveContainer" containerID="ed1fd7d34aed2eb2ad514c40d4ff506bb1a412471859d53e3d8faca54d3e1cb5" Nov 04 04:04:18.581433 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:18.581414 6000 scope.go:122] "RemoveContainer" containerID="ed1fd7d34aed2eb2ad514c40d4ff506bb1a412471859d53e3d8faca54d3e1cb5" Nov 04 04:04:18.581781 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:04:18.581732 6000 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed1fd7d34aed2eb2ad514c40d4ff506bb1a412471859d53e3d8faca54d3e1cb5\": container with ID starting with ed1fd7d34aed2eb2ad514c40d4ff506bb1a412471859d53e3d8faca54d3e1cb5 not found: ID does not exist" containerID="ed1fd7d34aed2eb2ad514c40d4ff506bb1a412471859d53e3d8faca54d3e1cb5" Nov 04 04:04:18.581927 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:18.581796 6000 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed1fd7d34aed2eb2ad514c40d4ff506bb1a412471859d53e3d8faca54d3e1cb5"} err="failed to get container status \"ed1fd7d34aed2eb2ad514c40d4ff506bb1a412471859d53e3d8faca54d3e1cb5\": rpc error: code = NotFound desc = could not find container \"ed1fd7d34aed2eb2ad514c40d4ff506bb1a412471859d53e3d8faca54d3e1cb5\": container with ID starting with ed1fd7d34aed2eb2ad514c40d4ff506bb1a412471859d53e3d8faca54d3e1cb5 not found: ID does not exist" Nov 04 04:04:18.592282 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:18.592252 6000 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf"] Nov 04 04:04:18.596569 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:18.596538 6000 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-gnswf"] Nov 04 04:04:18.939377 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:18.939294 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7vkwm_e0e8fde9-0afb-424f-af2a-4b93fb4fd6c8/node-ca/0.log" Nov 04 04:04:20.041770 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:20.041714 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-597d745469j8tv2_e8873cfa-cbb1-4b2b-aa61-fca152bee660/istio-proxy/0.log" Nov 04 04:04:20.083826 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:20.083793 6000 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="01a09535-2de8-4f06-846b-556b9db5e38b" path="/var/lib/kubelet/pods/01a09535-2de8-4f06-846b-556b9db5e38b/volumes" Nov 04 04:04:20.691741 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:20.691704 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-kkgr2_ee7b70f9-1025-4144-b882-fb75f1159c3e/istio-proxy/0.log" Nov 04 04:04:20.756165 ip-10-0-129-134 kubenswrapper[6000]: E1104 04:04:20.756126 6000 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rx686" containerName="extractor" Nov 04 04:04:20.767422 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:20.767395 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-skgsr_fa196927-03ec-4015-8172-44098857d262/istio-proxy/1.log" Nov 04 04:04:20.767609 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:20.767570 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-skgsr_fa196927-03ec-4015-8172-44098857d262/istio-proxy/0.log" Nov 04 04:04:21.546973 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:21.546934 6000 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-6pnpf" Nov 04 04:04:22.029340 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:22.029220 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c425l_ea2d0497-b79b-4640-b8ca-bd829b1f3dfa/serve-healthcheck-canary/0.log" Nov 04 04:04:22.898686 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:22.898657 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rx686_eff746dd-87c2-47fb-bee2-f6ebb6c11fc6/kube-rbac-proxy/0.log" Nov 04 04:04:22.930944 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:22.930914 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rx686_eff746dd-87c2-47fb-bee2-f6ebb6c11fc6/exporter/0.log" Nov 04 04:04:22.964091 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:22.964036 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rx686_eff746dd-87c2-47fb-bee2-f6ebb6c11fc6/extractor/0.log" Nov 04 04:04:24.915308 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:24.915270 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-597d745469j8tv2_e8873cfa-cbb1-4b2b-aa61-fca152bee660/istio-proxy/0.log" Nov 04 04:04:25.491777 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:25.490742 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-kkgr2_ee7b70f9-1025-4144-b882-fb75f1159c3e/istio-proxy/0.log" Nov 04 04:04:25.540981 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:25.540948 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-skgsr_fa196927-03ec-4015-8172-44098857d262/istio-proxy/1.log" Nov 04 04:04:25.541261 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:25.541246 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-skgsr_fa196927-03ec-4015-8172-44098857d262/istio-proxy/0.log" Nov 04 04:04:26.021129 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:26.021077 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-74f9c8978b-jtnnz_4493817e-586c-4ff4-ba4c-546b488b8925/manager/0.log" Nov 04 04:04:26.048696 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:26.048660 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-74f9c8978b-jtnnz_4493817e-586c-4ff4-ba4c-546b488b8925/copy-manifests/0.log" Nov 04 04:04:27.181085 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:27.181011 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-597d745469j8tv2_e8873cfa-cbb1-4b2b-aa61-fca152bee660/istio-proxy/0.log" Nov 04 04:04:27.758050 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:27.758015 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-kkgr2_ee7b70f9-1025-4144-b882-fb75f1159c3e/istio-proxy/0.log" Nov 04 04:04:27.809700 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:27.809669 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-skgsr_fa196927-03ec-4015-8172-44098857d262/istio-proxy/1.log" Nov 04 04:04:27.809885 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:27.809869 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-skgsr_fa196927-03ec-4015-8172-44098857d262/istio-proxy/0.log" Nov 04 04:04:28.378016 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:28.377974 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-5xmx7_6da524eb-f360-4de3-979e-d577dfae526d/manager/0.log" Nov 04 04:04:28.682556 ip-10-0-129-134 kubenswrapper[6000]: I1104 04:04:28.682465 6000 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-r66wn_0fa75f01-dc06-4030-a496-6a354acf20c2/proxy/0.log"